From owner-freebsd-net@FreeBSD.ORG Thu Nov 11 19:42:14 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 217D7106564A for ; Thu, 11 Nov 2010 19:42:14 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id A3BE28FC18 for ; Thu, 11 Nov 2010 19:42:13 +0000 (UTC) Received: by wyb36 with SMTP id 36so220070wyb.13 for ; Thu, 11 Nov 2010 11:42:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:content-type; bh=zvAgMhQ0oaJ1z33+GJ2UOQ++cV+vPrYpYCqGd3K1tTk=; b=qXBgH1obfqWEbMurrnI/c5UCjbXK/lD8P3g1HrNcdeUN1eTpZpm3QNIKgNAR10QiOC 4+bJHh1qxIeVy4QaGZr2whH2KRRpPSf2qXEmKeHY/OwV9LRlW7Qnucy7hooJYuU45Q2a 9lxdhx/MAmFR4JPz1FzXBDIi9Bl/pvrjWKBww= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=TShEsfRFG41+T6s54KnbyqgT+L9gG/XU2422AtY1v/qqG6UB/f/dQEnM99Bolc3G04 3+RPWGxRKSswAnfkJXAnXi/8p0s4e2GwWFo2X+tQuYdytyZAi8GS421c6Y+xb8JWkyf9 MumbB8pUqXlaHI2oIDJ2nJmU/n0iCi3xo81mc= MIME-Version: 1.0 Received: by 10.216.179.210 with SMTP id h60mr1138761wem.42.1289504532393; Thu, 11 Nov 2010 11:42:12 -0800 (PST) Received: by 10.216.2.206 with HTTP; Thu, 11 Nov 2010 11:42:12 -0800 (PST) In-Reply-To: <20101111180539.GC11275@traktor.dnepro.net> References: <20101110110428.GA3505@traktor.dnepro.net> <20101111104952.GA11275@traktor.dnepro.net> <20101111180539.GC11275@traktor.dnepro.net> Date: Thu, 11 Nov 2010 11:42:12 -0800 Message-ID: From: Jack Vogel To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: igb dual-port adapter 1200Mbps limit - what to tune? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Nov 2010 19:42:14 -0000 The driver already handles the pinning, you shouldnt need to mess with it. MSIX interrupts start at 256, the igb driver uses one vector per queue, which is an TX/RX pair. The driver creates as many queues as cores up to a max of 8. Jack On Thu, Nov 11, 2010 at 10:05 AM, Eugene Perevyazko wrote: > On Thu, Nov 11, 2010 at 12:49:52PM +0200, Eugene Perevyazko wrote: > > On Thu, Nov 11, 2010 at 01:47:02AM +0100, Ivan Voras wrote: > > > On 11/10/10 12:04, Eugene Perevyazko wrote: > > > > > > >Tried 2 queues and 1 queue per iface, neither hitting cpu limit. > > > > > > Are you sure you are not hitting the CPU limit on individual cores? > Have > > > you tried running "top -H -S"? > > > > > Sure, even with 1queue per iface load is 40-60% on busy core, with 2 > queues it was much lower. > > Now I've got the module for mb with 2 more ports, going to see if it > helps. > The IO module has em interfaces on it and somehow I've already got 2 panics > after moving one of vlans to it. > > In the mean time, can someone explain me what is processed by threads > marked > like "irq256: igb0" and "igb0 que". May be understanding this will let me > pin those threads to cores more optimally. > There are (hw.igb.num_queues+1) "irq" threads and (hw.igb.num_queues) "que" > threads. Now I just pin them sequentially to even cores (odd ones are HT). > > Now I use hw.igb.num_queues=2, and with traffic limited to 1200Mbits the > busiest core is still 60% idle... > > > > -- > Eugene Perevyazko > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >