From owner-freebsd-net@FreeBSD.ORG Thu May 19 16:37:56 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F32DD1065673 for ; Thu, 19 May 2011 16:37:56 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id A8E0D8FC16 for ; Thu, 19 May 2011 16:37:56 +0000 (UTC) Received: by vws18 with SMTP id 18so2801630vws.13 for ; Thu, 19 May 2011 09:37:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=R7cLkFm27VA7Rfkf15cb16IaR2LqEi9vuufUuOehYYc=; b=TyRFclh4G+/ZVCmC1MXkHtQeMV47kJJ+YWak/NYkKf6JcXvgniomxd63lJLC96N/yU JtpHmeK5WDUAQRd4b6hTP0dCCninqdIz03AuZK5sjO4n05L9oyDTgqECD9dxgfgh/ELk yt+TDeY0jphXdZ3e3XnRQHVGfFQ6D6Lwbtrc8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=xYwK6xndLz+M3f51/4iFxhOL60Sb+lELhQQug69hcJ2ykMI3/YFChbLwE2MRPgTsSn gm6H1js+goZDwnpUwQbrkDh6DxIeyjLZ9u/Cy1MpFqG3C7z8QChBw4O67GUjyJtCbeGj AZfc3C6hPEku4DwkjH1Lx4el4kUCOkVcnNrCk= MIME-Version: 1.0 Received: by 10.52.179.103 with SMTP id df7mr4947426vdc.148.1305823063789; Thu, 19 May 2011 09:37:43 -0700 (PDT) Received: by 10.52.183.233 with HTTP; Thu, 19 May 2011 09:37:43 -0700 (PDT) In-Reply-To: References: Date: Thu, 19 May 2011 09:37:43 -0700 Message-ID: From: Jack Vogel To: William Salt Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-net@freebsd.org Subject: Re: Intel 10GbE Tuning under freebsd X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 May 2011 16:37:57 -0000 First, get the latest driver if you are using the native 8.1 its way old. Second, make sure you're in a PCI Express 2.0 slot, at least 8x, and if you are using a consumer type system there are often slots that are not wired with as many lanes as you might think, so verify that. Oh, and remember, both sides of the link effect things so check both. Newer driver versions will warn you if you don't have adequate bandwidth. Third, make sure you have the system interrupt storm threshold increased, hw.intr_storm_threshold, its way too low by default for 10G rates, I would initially increase it to 10000, there will be messages about throttling if its not high enough, increase til you don't see them. Fourth, make sure your mbuf pool numbers are high enough in the size that you need. The ixgbe driver uses standard 2K clusters when using standard MTU, but as soon as you use jumbos it will go to 4K and then 9K as the jumbo size increases. increase the relevant kern.ipc.nmb... This should get you in the ballpark, let me know if you have further questions or issues. Good luck, Jack On Thu, May 19, 2011 at 6:52 AM, William Salt wrote: > Hi All, > I have just got a couple of 10GbE intel X520-DA2 cards to test. Im > running freebsd 8.1 on a super micro intel xeon server, and a hp core 2 duo > workstation, both machines have 4gb of ram. > Both are attached via sfp+ cables to a brocade turbo iron switch, which > simply has jumbo frames enabled. > > Initial testing was pretty basic, i simply enabled jumbo frames, and > increased max send/recv buffer to 16MB, which should be more than enough. > I then ran iperf and managed to get approximately 5gbps, i was expecting > around 8.5gbps. > However, i realise that the ix driver probably needs to be tuned, as well > as > the OS. > > After a bit of googling, i cant find a tuning guide under freebsd for > 10GbE, > or many recommendations, and wondered if anyone has already successfully > managed to tune freebsd with intel 10gbe nics, to gain a higher throughput? > If so, has anyone got any tips, or sample configs? > > Thanks in advance > Will > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >