From owner-freebsd-net@FreeBSD.ORG Tue Nov 24 19:40:57 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8F15D106566C for ; Tue, 24 Nov 2009 19:40:57 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from qw-out-2122.google.com (qw-out-2122.google.com [74.125.92.24]) by mx1.freebsd.org (Postfix) with ESMTP id 41F768FC14 for ; Tue, 24 Nov 2009 19:40:56 +0000 (UTC) Received: by qw-out-2122.google.com with SMTP id 9so1544147qwb.7 for ; Tue, 24 Nov 2009 11:40:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:date:to:cc :subject:message-id:reply-to:references:mime-version:content-type :content-disposition:in-reply-to:user-agent; bh=c/6v1vYjtmC5awCMMvTNi9uAtPDqNVpbqIQwYDydjmg=; b=TtbCZi8ctiZsZx7VrTfcNwfVP8rLA2BDhnR9E9cUHKPZnW8Yss02jNgV1FaTew6YPu Vgx3ofKs54yECBdMdmvOuf74DiK3V7sm9Cv+1AtlGiHcexaMN3c1j2eAkHKzM1ou6Sp9 EmQnY+UOJotxpIEW8gfHbkNM2jzN/0VUPMUNE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=aI//vh/Cns9cnZoHHLc9QVvhvzH75USt4eG5p9rWPz4yq9WLVhgNErNm5q2Dj6XGqC Gdf3/P/fleFuevfju3YzEfCKcE8VCV2yEwUp2qD8i3oS2YOTSTw5DjqpIN/O3+HeQ+Sv HhmR5cutmrxJqJkApavryNanWT9PnmtrEgFck= Received: by 10.224.105.26 with SMTP id r26mr3430374qao.0.1259091656325; Tue, 24 Nov 2009 11:40:56 -0800 (PST) Received: from pyunyh@gmail.com ([174.35.1.224]) by mx.google.com with ESMTPS id 7sm16081684qwb.2.2009.11.24.11.40.54 (version=TLSv1/SSLv3 cipher=RC4-MD5); Tue, 24 Nov 2009 11:40:55 -0800 (PST) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Tue, 24 Nov 2009 11:40:25 -0800 From: Pyun YongHyeon Date: Tue, 24 Nov 2009 11:40:25 -0800 To: "Yuriy A. Korobko" Message-ID: <20091124194025.GA1116@michelle.cdnetworks.com> References: <1258998140.3015.34.camel@stormi-desktop> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1258998140.3015.34.camel@stormi-desktop> User-Agent: Mutt/1.4.2.3i Cc: freebsd-net@freebsd.org Subject: Re: Reducing number of interrupts from intel pro 1000 et adapter X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Nov 2009 19:40:57 -0000 On Mon, Nov 23, 2009 at 07:42:20PM +0200, Yuriy A. Korobko wrote: > Hi, > > I'd like to know a way to control tx interrupts on intel pro 1000 et > adapter with igb driver. Just installed one in the router and systat > shows 8-9k rx interrupts and 20k tx interrupts from igb0 and igb1 > adapters. Box is a router running freebsd 7.2 release, I've tried > default driver from kernel source and latest from intel site, effect is > the same with automatic interrupt moderation enabled and disabled. I I'm also aware of this issue. Here is patch I'm currently experimenting. It seems igb(4) wants to dynamically adjust interrupt moderation on Rx traffic such that this seems to cause lots of Tx interrupts under heavy Rx traffic. I simply disabled that feature and fixed Rx handler not to generate more interrupts. Without Rx handler fix the igb(4) took all CPU cycles under heady load(64 bytes UDP torture test) I couldn't even type a character on console. You can get my patch at the following URL. http://people.freebsd.org/~yongari/igb/igb.buf.patch5 The patch also includes other changes I made so it's somewhat big. Note, please don't apply the patch on production servers it needs more testing. > have the same box with intel pro 1000 pt adapter which have > tx(rx)_int_delay sysctls in em driver, I was able to reduce number of > tx/rx interrupts to 7-8k per interface and got much more cpu idle > because of less context switches with same pps. > I guess you sacrificed latencies.