From owner-freebsd-net@FreeBSD.ORG Fri Apr 25 11:37:06 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E830FB5D for ; Fri, 25 Apr 2014 11:37:06 +0000 (UTC) Received: from mail.blacquiere.nl (provider.blacquiere.nl [144.76.110.43]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 92D391685 for ; Fri, 25 Apr 2014 11:37:05 +0000 (UTC) Received: from calendar.blacquiere.nl ([192.168.201.5] helo=shell.blacquiere.nl) by mail.blacquiere.nl with esmtps (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.80.1 (FreeBSD)) (envelope-from ) id 1Wde4J-000CZs-Bn; Fri, 25 Apr 2014 13:13:43 +0200 Date: Fri, 25 Apr 2014 13:13:16 +0200 From: Robert Blacquiere To: freebsd-net@freebsd.org Message-ID: <20140425111316.GJ9177@calendar.blacquiere.nl> References: <535A1354.2040309@wp.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <535A1354.2040309@wp.pl> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam_score: -4.4 X-Spam_score_int: -43 X-Spam_bar: ---- X-Spam_report: Spam detection software, running on the system "mail", has identified this incoming email as possible spam. The original message has been attached to this so you can view it (if it isn't spam) or label similar future email. If you have any questions, see The administrator of that system for details. Content preview: Hi Marek, On Fri, Apr 25, 2014 at 09:48:36AM +0200, Marek Salwerowicz wrote: > Hi list, > > Both boxes are connected to the same switch (HP 1910-48G) > > I need to transfer around 10 TB of data from storage1 to storage2 > I obeserve that during copying, only one NIC (instead of all 4) is used. > > Boxes are not stressed during copying > > What's more, apart from having 1 NIC saturated (transfer around 120 > MB/s), I observe transfer rate on level of 70-80 MB/s [...] Content analysis details: (-4.4 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP -1.5 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] X-SA-Exim-Connect-IP: 192.168.201.5 X-SA-Exim-Mail-From: robert@blacquiere.nl X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on mail X-Spam-Level: X-Spam-Status: No, score=-4.4 required=5.0 tests=ALL_TRUSTED,BAYES_00, RP_MATCHES_RCVD autolearn=ham version=3.3.2 Subject: Re: NFS over LAGG / lacp poor performance X-SA-Exim-Version: 4.2 X-SA-Exim-Scanned: Yes (on mail.blacquiere.nl) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 11:37:07 -0000 Hi Marek, On Fri, Apr 25, 2014 at 09:48:36AM +0200, Marek Salwerowicz wrote: > Hi list, > > Both boxes are connected to the same switch (HP 1910-48G) > > I need to transfer around 10 TB of data from storage1 to storage2 > I obeserve that during copying, only one NIC (instead of all 4) is used. > > Boxes are not stressed during copying > > What's more, apart from having 1 NIC saturated (transfer around 120 > MB/s), I observe transfer rate on level of 70-80 MB/s Default lacp 802.3ad works with mac based hashes to loadbalance traffic. So single host (mac) will be transfered by one ethernet adaptor as you have seen. > > I haven't changed any kernel configuration, nor sysctl's - am I missing > sth ? There are tuneables, i think, to change the hash from mac source : dest hash to session tcp source+port: dest:port but these might need also configuring on switch/network. Regards Robert Blacquiere