From owner-freebsd-net@FreeBSD.ORG Fri Apr 25 15:52:43 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7F946AB4 for ; Fri, 25 Apr 2014 15:52:43 +0000 (UTC) Received: from umail.aei.mpg.de (umail.aei.mpg.de [194.94.224.6]) by mx1.freebsd.org (Postfix) with ESMTP id 0D87314C6 for ; Fri, 25 Apr 2014 15:52:43 +0000 (UTC) Received: from mailgate.aei.mpg.de (mailgate.aei.mpg.de [194.94.224.5]) by umail.aei.mpg.de (Postfix) with ESMTP id D44BB201110; Fri, 25 Apr 2014 17:52:41 +0200 (CEST) Received: from mailgate.aei.mpg.de (localhost [127.0.0.1]) by localhost (Postfix) with SMTP id C7C7B405889; Fri, 25 Apr 2014 17:52:41 +0200 (CEST) Received: from intranet.aei.uni-hannover.de (ahin1.aei.uni-hannover.de [130.75.117.40]) by mailgate.aei.mpg.de (Postfix) with ESMTP id A3A50406AF1; Fri, 25 Apr 2014 17:52:41 +0200 (CEST) Received: from cascade.aei.uni-hannover.de ([10.117.15.111]) by intranet.aei.uni-hannover.de (Lotus Domino Release 8.5.3FP6) with ESMTP id 2014042517523117-88837 ; Fri, 25 Apr 2014 17:52:31 +0200 Date: Fri, 25 Apr 2014 17:52:31 +0200 From: Gerrit =?ISO-8859-1?Q?K=FChn?= To: "Steven Hartland" Subject: Re: NFS over LAGG / lacp poor performance Message-Id: <20140425175231.9714afc3.gerrit.kuehn@aei.mpg.de> In-Reply-To: <006183CE786F4EAC9435B797E5F271CF@multiplay.co.uk> References: <535A1354.2040309@wp.pl> <20140425113711.e7c7d1c2.gerrit.kuehn@aei.mpg.de> <535A482E.1030106@wp.pl> <20140425140123.a76c18f9.gerrit.kuehn@aei.mpg.de> <535A5268.100@wp.pl> <8247FE6336414E1F97ADA561D0680097@multiplay.co.uk> <535A5DD9.9060206@wp.pl> <535A69CE.9010800@wp.pl> <4E7A280CACC84910A3DE068085976E30@multiplay.co.uk> <20140425161312.3ad95e3a.gerrit.kuehn@aei.mpg.de> <1BD2C00D4F0D405C939C975246D41C60@multiplay.co.uk> <20140425163433.e35e5de3.gerrit.kuehn@aei.mpg.de> <006183CE786F4EAC9435B797E5F271CF@multiplay.co.uk> Organization: Max Planck Gesellschaft X-Mailer: Sylpheed 3.1.3 (GTK+ 2.24.19; amd64-portbld-freebsd8.2) Mime-Version: 1.0 X-MIMETrack: Itemize by SMTP Server on intranet/aei-hannover(Release 8.5.3FP6|November 21, 2013) at 04/25/2014 17:52:31, Serialize by Router on intranet/aei-hannover(Release 8.5.3FP6|November 21, 2013) at 04/25/2014 17:52:40, Serialize complete at 04/25/2014 17:52:40 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-PMX-Version: 6.0.2.2308539, Antispam-Engine: 2.7.2.2107409, Antispam-Data: 2014.4.25.153920 X-PerlMx-Spam: Gauge=IIIIIIII, Probability=8%, Report=' HTML_00_01 0.05, HTML_00_10 0.05, MIME_LOWER_CASE 0.05, BODYTEXTP_SIZE_3000_LESS 0, BODY_SIZE_1800_1899 0, BODY_SIZE_2000_LESS 0, BODY_SIZE_5000_LESS 0, BODY_SIZE_7000_LESS 0, __ANY_URI 0, __BOUNCE_CHALLENGE_SUBJ 0, __BOUNCE_NDR_SUBJ_EXEMPT 0, __CT 0, __CTE 0, __CT_TEXT_PLAIN 0, __HAS_FROM 0, __HAS_MSGID 0, __HAS_X_MAILER 0, __IN_REP_TO 0, __MIME_TEXT_ONLY 0, __MIME_VERSION 0, __SANE_MSGID 0, __STOCK_PHRASE_7 0, __SUBJ_ALPHA_NEGATE 0, __TO_MALFORMED_2 0, __URI_NO_PATH 0, __URI_NO_WWW 0, __URI_NS ' Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 15:52:43 -0000 On Fri, 25 Apr 2014 15:55:29 +0100 "Steven Hartland" wrote about Re: NFS over LAGG / lacp poor performance: SH> Run top then enable system processes (shift+S) and threads (shift+H) SH> then disable idle process display (z) if your seeing significant CPU SH> usage to the igb interrupt handlers try reducing the number. Hm, I'm a bit unsure what I see here: --- last pid: 71026; load averages: 0.36, 0.37, 0.27 up 18+03:14:21 17:48:02 295 processes: 5 running, 259 sleeping, 31 waiting CPU: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle Mem: 3596K Active, 59M Inact, 11G Wired, 900K Cache, 1236M Buf, 1019M Free Swap: 16G Total, 14M Used, 16G Free PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND 0 root -16 0 0K 2672K - 2 272:01 0.00% kernel{zio_write_issue_} 0 root -16 0 0K 2672K - 0 271:55 0.00% kernel{zio_write_issue_} 0 root -16 0 0K 2672K - 3 271:54 0.00% kernel{zio_write_issue_} 0 root -16 0 0K 2672K - 1 271:52 0.00% kernel{zio_write_issue_} 1455 root 20 0 9912K 472K rpcsvc 2 173:53 0.00% nfsd{nfsd: master} 1455 root 20 0 9912K 472K rpcsvc 0 166:37 0.00% nfsd{nfsd: service} 1455 root 20 0 9912K 472K rpcsvc 0 164:25 0.00% nfsd{nfsd: service} 1455 root 20 0 9912K 472K rpcsvc 3 163:06 0.00% nfsd{nfsd: service} 12 root -92 - 0K 496K WAIT 1 155:36 0.00% intr{irq262: igb1:que} [...] --- The irq-igb1 clibs to top every now and then (always less than 1% cpu usage though), but there is never much cpu usage shown for any task. However, the overall load is significant. What might the system be doing? cu Gerrit