From owner-freebsd-net@FreeBSD.ORG Thu Feb 2 06:35:31 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 35A84106564A for ; Thu, 2 Feb 2012 06:35:31 +0000 (UTC) (envelope-from andrey@zonov.org) Received: from mail-bk0-f54.google.com (mail-bk0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id BC3C28FC0C for ; Thu, 2 Feb 2012 06:35:30 +0000 (UTC) Received: by bkbzx1 with SMTP id zx1so2406586bkb.13 for ; Wed, 01 Feb 2012 22:35:29 -0800 (PST) Received: by 10.204.150.69 with SMTP id x5mr724271bkv.53.1328164529078; Wed, 01 Feb 2012 22:35:29 -0800 (PST) Received: from [10.254.254.77] (ppp95-165-140-229.pppoe.spdop.ru. [95.165.140.229]) by mx.google.com with ESMTPS id ek9sm3995560bkb.10.2012.02.01.22.35.28 (version=SSLv3 cipher=OTHER); Wed, 01 Feb 2012 22:35:28 -0800 (PST) Message-ID: <4F2A2EAB.3010700@zonov.org> Date: Thu, 02 Feb 2012 10:35:23 +0400 From: Andrey Zonov User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; ru; rv:1.8.1.24) Gecko/20100228 Thunderbird/2.0.0.24 Mnenhy/0.7.6.0 MIME-Version: 1.0 To: "Alexander V. Chernikov" References: <4F29A464.3080302@zonov.org> <4F29E2C8.5000909@FreeBSD.org> In-Reply-To: <4F29E2C8.5000909@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org Subject: Re: netisr defered - active only one thread X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Feb 2012 06:35:31 -0000 On 02.02.2012 5:11, Alexander V. Chernikov wrote: > On 01.02.2012 20:45, Andrey Zonov wrote: >> Hi, >> >> I'm trying to tune machine with 8.2-STABLE for heavy network load and >> now playing with netisr. Could anyone explain me why actually works only >> one netisr thread if I set them to 8? > > Can you please supply `nestat -Q` output and clarify you usage pattern ? > (I mean, this is router/web server/some kind of traffic receiver/etc..). > For example, flow policy does not balance traffic from single flow > between different CPUs. > This is a web server with multiple nginx instances. 5k/sec accepted connections. Input packet rate is 35kpps, output - 25kpps. I thought of changing policy for IP, but how can I do this (without patching)? Is it safe? netstat -Q (I turned on direct & direct force for now): Configuration: Setting Value Maximum Thread count 8 8 Default queue limit 256 10240 Direct dispatch enabled n/a Forced direct dispatch enabled n/a Threads bound to CPUs enabled n/a Protocols: Name Proto QLimit Policy Flags ip 1 5000 flow --- igmp 2 256 source --- rtsock 3 256 source --- arp 7 256 source --- ip6 10 256 flow --- Workstreams: WSID CPU Name Len WMark Disp'd HDisp'd QDrops Queued Handled 0 0 ip 0 0 1125716 0 0 0 1125716 igmp 0 0 0 0 0 0 0 rtsock 0 1 0 0 0 102 102 arp 0 0 27 0 0 0 27 ip6 0 0 0 0 0 0 0 1 1 ip 0 0 1222701 0 0 0 1222701 igmp 0 0 0 0 0 0 0 rtsock 0 0 0 0 0 0 0 arp 0 0 46 0 0 0 46 ip6 0 0 0 0 0 0 0 2 2 ip 0 0 1184381 0 0 0 1184381 igmp 0 0 0 0 0 0 0 rtsock 0 0 0 0 0 0 0 arp 0 0 45 0 0 0 45 ip6 0 0 0 0 0 0 0 3 3 ip 0 0 1191094 0 0 0 1191094 igmp 0 0 0 0 0 0 0 rtsock 0 0 0 0 0 0 0 arp 0 0 54 0 0 0 54 ip6 0 0 0 0 0 0 0 4 4 ip 0 0 846165 0 0 0 846165 igmp 0 0 0 0 0 0 0 rtsock 0 0 0 0 0 0 0 arp 0 0 19 0 0 0 19 ip6 0 0 0 0 0 0 0 5 5 ip 0 0 849478 0 0 0 849478 igmp 0 0 0 0 0 0 0 rtsock 0 0 0 0 0 0 0 arp 0 0 27 0 0 0 27 ip6 0 0 0 0 0 0 0 6 6 ip 0 0 870836 0 0 0 870836 igmp 0 0 0 0 0 0 0 rtsock 0 0 0 0 0 0 0 arp 0 0 29 0 0 0 29 ip6 0 0 0 0 0 0 0 7 7 ip 0 5000 594320 5 910862 3453459 4047784 igmp 0 0 0 0 0 0 0 rtsock 0 0 0 0 0 0 0 arp 0 5 21 0 0 109 130 ip6 0 1 0 0 0 1 1 -- Andrey Zonov