From owner-freebsd-net@FreeBSD.ORG Fri Apr 25 14:55:31 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AA16F381 for ; Fri, 25 Apr 2014 14:55:31 +0000 (UTC) Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35]) by mx1.freebsd.org (Postfix) with ESMTP id 67A9E1D25 for ; Fri, 25 Apr 2014 14:55:31 +0000 (UTC) Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534) id AB34120E7088B; Fri, 25 Apr 2014 14:55:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.multiplay.co.uk X-Spam-Level: ** X-Spam-Status: No, score=2.2 required=8.0 tests=AWL,BAYES_00,DOS_OE_TO_MX, FSL_HELO_NON_FQDN_1,HELO_NO_DOMAIN,RDNS_DYNAMIC,STOX_REPLY_TYPE autolearn=no version=3.3.1 Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170]) by smtp1.multiplay.co.uk (Postfix) with ESMTPS id D438A20E70885; Fri, 25 Apr 2014 14:55:25 +0000 (UTC) Message-ID: <006183CE786F4EAC9435B797E5F271CF@multiplay.co.uk> From: "Steven Hartland" To: =?iso-8859-1?Q?Gerrit_K=FChn?= References: <535A1354.2040309@wp.pl><20140425113711.e7c7d1c2.gerrit.kuehn@aei.mpg.de><535A482E.1030106@wp.pl><20140425140123.a76c18f9.gerrit.kuehn@aei.mpg.de><535A5268.100@wp.pl><8247FE6336414E1F97ADA561D0680097@multiplay.co.uk><535A5DD9.9060206@wp.pl><535A69CE.9010800@wp.pl><4E7A280CACC84910A3DE068085976E30@multiplay.co.uk><20140425161312.3ad95e3a.gerrit.kuehn@aei.mpg.de><1BD2C00D4F0D405C939C975246D41C60@multiplay.co.uk> <20140425163433.e35e5de3.gerrit.kuehn@aei.mpg.de> Subject: Re: NFS over LAGG / lacp poor performance Date: Fri, 25 Apr 2014 15:55:29 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 8bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 14:55:31 -0000 ----- Original Message ----- From: "Gerrit Kühn" > On Fri, 25 Apr 2014 15:25:04 +0100 "Steven Hartland" > wrote about Re: NFS over LAGG / lacp poor > performance: > > SH> We saw the issue with 10-RELEASE last weekend on a machine with 6 x > SH> igb's where by the system was burning CPU in the interrupt handlers, > SH> setting num_queues to 2 fixed the issue we where seeing. > > Sounds like reducing the queues is the first thing to try then, thanks! > How did you verify that the irqs were the culprit? Apart from the high irq > rate in vmstat I do not see any other load values skyrocketing. It was more CPU usage in top. Run top then enable system processes (shift+S) and threads (shift+H) then disable idle process display (z) if your seeing significant CPU usage to the igb interrupt handlers try reducing the number. Regards Steve