From owner-freebsd-net@FreeBSD.ORG Wed Dec 1 18:09:32 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 99ABE10656A3 for ; Wed, 1 Dec 2010 18:09:32 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 25D278FC08 for ; Wed, 1 Dec 2010 18:09:31 +0000 (UTC) Received: by wwf26 with SMTP id 26so3008859wwf.31 for ; Wed, 01 Dec 2010 10:09:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=j062RVf/NmFyfuAoLuPSy1FHTiNnQBzoAkNyo7aZx+s=; b=IBHKDlQayFYJ8ZcflTo4JJ2K8qvg8kDYCt1ELZwTVZgT92kuqfnuJU2BF1zvAE7TKN vzIPWaEgQQPBTXpRDIngoK7No/2HO8sX/HOszRIfEcxEXrvMBx3LVf5TWjvh3Rh4WOmn kEpX7wO4+cMk7Yk9NKWTDSLb4Y50ZaqoTdpII= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=VQeEMaqCG7JzVSHy2A7ia2lqQGbLRiPyfA+WY7keHn7+ep5idmlO4e3nPnhZWnQ8+T K1l3bulquNh07+3Zi3OCaObr5yPJwZ0r4zW2/uiPvgDzRzafy9BJLhvZUdcic0ZpKQwv xq3U3S3uIONPheyYpAFHJhvwUk+/jFBKbk9a0= MIME-Version: 1.0 Received: by 10.216.59.193 with SMTP id s43mr2366181wec.42.1291226971149; Wed, 01 Dec 2010 10:09:31 -0800 (PST) Received: by 10.216.2.206 with HTTP; Wed, 1 Dec 2010 10:09:31 -0800 (PST) In-Reply-To: References: <20101110110428.GA3505@traktor.dnepro.net> <20101201175401.GA69269@traktor.dnepro.net> Date: Wed, 1 Dec 2010 10:09:31 -0800 Message-ID: From: Jack Vogel To: Vlad Galu Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-net@freebsd.org Subject: Re: igb dual-port adapter 1200Mbps limit - what to tune? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Dec 2010 18:09:32 -0000 And why don't you use the driver in 7.4 PRERELEASE (2.0.7) and eliminate any inevitable questions about fixes you don't have. Jack On Wed, Dec 1, 2010 at 10:06 AM, Vlad Galu wrote: > On Wed, Dec 1, 2010 at 6:54 PM, Eugene Perevyazko wrote: > > On Wed, Nov 10, 2010 at 01:04:28PM +0200, Eugene Perevyazko wrote: > >> > >> I have a router running RELENG_7 with two dual-port igbs - igb0 and igb1 > are on > >> 82575 on intel s5520ur mb and igb2 and igb3 are on 82576 on ET dual-port > card. > >> 82576 is in 8x slot. > >> Main traffic flows from igb0+igb1 to igb2+igb3, less traffic goes back. > >> There's no traffic flow in directions igb0 - igb1 and igb2 - igb3. > >> > >> There are vlans on all interfaces. > >> > >> igb0 and igb1 are outbound links. > >> igb2 and igb3 are connected to switch. > >> CPU is E5620@2.4GHz, 8 cores, irqs bound to different cores skipping HT > ones. > >> Tried 2 queues and 1 queue per iface, neither hitting cpu limit. > >> > >> The problem is that traffic through igb2+igb3 is limited at around > 1200Mbps Tx > >> while I was hoping for 1600-1800Mbps Tx. > >> > > > > I'd like to say that now this host is forwarding 1710 Mb/s @ 189 kpps in > one direction + 411 Mb/s @ 140 kpps in reverse direction (30 minutes > average). > > The following changes were made: > > - no lagg in use > > - only one vlan left instead of 5 vlans > > - added motherboard module 'HP NC360T PCIe DP Gigabit Server Adapter > (n1e5132)' with 2 em interfaces (only one used) > > - 3 interfaces are connected to hosts with patchcords, not through > switch > > - hyperthreading turned off in bios > > - igb driver patched to reduce irq rate and make it tunable (patch > follows) > > - hw.igb.num_queues=4, dev.igb.*.enable_aim=0, > dev.igb.*.default_intrrate=4000 the last sysctl is added in patch) > > - igb queues manually repinned to different cores > > - ipfw rules minimized > > Can you please run a test with IPFW disabled? It's best to avoid using > a firewall where throughput is the primary goal. > > > > > igb driver used is 1.9.6/RELENG_7, igb.c version 1.3.2.12 > > > > Can't say for sure whether there is a key change or is it all changes > summed up, > > sorry. > > > > -- > > Eugene Perevyazko > > > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > > > > > -- > Good, fast & cheap. Pick any two. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >