From owner-freebsd-current@FreeBSD.ORG Fri Jul 20 23:10:05 2007 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0D3A816A418 for ; Fri, 20 Jul 2007 23:10:05 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from wa-out-1112.google.com (wa-out-1112.google.com [209.85.146.180]) by mx1.freebsd.org (Postfix) with ESMTP id D25CD13C467 for ; Fri, 20 Jul 2007 23:10:04 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: by wa-out-1112.google.com with SMTP id j37so1223702waf for ; Fri, 20 Jul 2007 16:10:04 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=mMBMPvgytRxirc/VeRyF2Uo1g251/7lGxlkYlpT12IRRe1m6ZAVZIaH9L4P/Gqe3UmmPRdJ0YPSqiJymQMDohHByy0mgIsJoE3JGpBLhpK7PPzgRuKq1YhXE/0TN6MUBtm0TQXvNJkAblxK7agOhctvjoaSMWno+UaE3jJO2WZw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=QshCue9c4oQuGKsuWqo72Az25cv64o6WkNjZwOZye2KQjXAEETRcSs0Mp6YIWpvDZV7rtoGAqOeiSauqxyhwLGu6dracDjbXDUAaMUGypgb2UuYduVZhPGlotsbkQJkzvOM12B/Sor4GSAtprYoOCSlvD7s0Tx2FuZzvU/UnYR4= Received: by 10.115.109.1 with SMTP id l1mr906085wam.1184973004534; Fri, 20 Jul 2007 16:10:04 -0700 (PDT) Received: by 10.114.103.14 with HTTP; Fri, 20 Jul 2007 16:10:04 -0700 (PDT) Message-ID: <2a41acea0707201610o48ed9b20ub3d22a0206982b19@mail.gmail.com> Date: Fri, 20 Jul 2007 16:10:04 -0700 From: "Jack Vogel" To: "Alexandre Biancalana" In-Reply-To: <2a41acea0707201600x3ae07833m1a54b2e5ae3d1b7c@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <20070716190441.GA19282@eschew.pusen.org> <20070716213425.GB19282@eschew.pusen.org> <20070718041159.GC37935@cdnetworks.co.kr> <20070718044700.GE37935@cdnetworks.co.kr> <20070718124350.GA25799@eschew.pusen.org> <46A06D8D.1070604@unsane.co.uk> <20070720122443.GA29372@eschew.pusen.org> <2a41acea0707200810n21843c76s9b0f4f37ef92722f@mail.gmail.com> <8e10486b0707201548r18aabeddt9655ed5372a1aaa7@mail.gmail.com> <2a41acea0707201600x3ae07833m1a54b2e5ae3d1b7c@mail.gmail.com> Cc: =?ISO-8859-1?Q?St=E5le_Kristoffersen?= , Vince , =?ISO-8859-1?Q?St=E5le_Kristoffersen?= , current@freebsd.org Subject: Re: Slow networkperformance in current? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jul 2007 23:10:05 -0000 On 7/20/07, Jack Vogel wrote: > On 7/20/07, Alexandre Biancalana wrote: > > On 7/20/07, Jack Vogel wrote: > > > > > > I am actually quite impressed with the stack right now, with our new > > > Oplin hardware and the ixgbe driver I have 9.8 Gb/s pumping thru > > > it. Of course its all about adequate hardware, configuration, and > > > tuning. > > > > > > Can you post the configuration tuning done ? > > > > > > Get rid of INVARIANT and WITNESS of course. > Use ULE scheduler > Use MTU of 9000 > Set interrupt storm threshold up to 8000 (default is 1000), if we didnt > do this the channel would get throttled. > > Then we need to make sure MSI/X is enabled, and set the number > of RX queues to 8 (this is Oplin specific). > > The ixgbe driver uses 4K jumbo clusters as long as you set MTU > up. I have experimented with Kip's patch for 9K and it helps a bit > but mainly it improves cpu utilization. > > Beyond this doing 10G is very sensitive to things like cpu's, frontside bus, > memory, etc, but these are things that I didn't personally set up, I just > tell them to give me a kick ass machine :) OH, one other item I forgot, use TSO, I think thats significant help.