From owner-freebsd-net@FreeBSD.ORG Mon Sep 14 08:58:41 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2A6A01065679 for ; Mon, 14 Sep 2009 08:58:41 +0000 (UTC) (envelope-from barney_cordoba@yahoo.com) Received: from n3b.bullet.mail.ac4.yahoo.com (n3b.bullet.mail.ac4.yahoo.com [76.13.13.73]) by mx1.freebsd.org (Postfix) with SMTP id C419C8FC1E for ; Mon, 14 Sep 2009 08:58:40 +0000 (UTC) Received: from [76.13.13.25] by n3.bullet.mail.ac4.yahoo.com with NNFMP; 14 Sep 2009 08:58:39 -0000 Received: from [76.13.10.183] by t4.bullet.mail.ac4.yahoo.com with NNFMP; 14 Sep 2009 08:58:39 -0000 Received: from [127.0.0.1] by omp124.mail.ac4.yahoo.com with NNFMP; 14 Sep 2009 08:58:39 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 789271.88459.bm@omp124.mail.ac4.yahoo.com Received: (qmail 41415 invoked by uid 60001); 14 Sep 2009 08:58:39 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1252918719; bh=DNhEQlbOOR9OBa/ekClyZPA5OCTpiSBxfgq/fQ8UkWI=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=3zJnuHQgvjg6auunhdqRznNpUpZq7VCODxKQDZaiOFEN/Do/AP4rpNlqhMlQNz2XG0Ef5dT4vrLm7TJizljGs3Tz8BX4/AH6TZZobQgYOSsLW2I8aqxQp0uIFisjf5ec2zcNpGsnC6FAf8qZRaOMCpeX+S16UukxYYMokgoZFow= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=5QhZHNutlzznh1F8E2b0q19G8C506bXIOYbr71yVoQCIjyXn5BSU6zzUHjswAMX68xNo13b5x2+7sOqIHmM8WDRXT54nxBimXd13nA5f9KAoy3l/8hUYN3GZWTHH5EyeeIS8hrW2gaDDjn8yQzq4SOE+r728T5oyIor5cQoMX1k=; Message-ID: <676338.40771.qm@web63905.mail.re1.yahoo.com> X-YMail-OSG: Npi1nUQVM1mDJopRUGuiFbnCAGqmAJrhRNB8ozmoOfLeGXrqGV5tg_lij_lmoNrnml7vJjeSGBahuVwn0Xtpkzuq9scXmcUwdBLRK_M9qIkEdml8L1SbyYK1HTIYdI.lUL9XrgskO3hhMAgwq.5hmIdPzdlRlVs7yQK96oXy9SkTF3d1A5.4GW2IUvPb3f.5zCcJyI1_62aptgRePRFxDSxCdA7ByiQqpuwfz8yrd9VDohiUw2alrdDPbgBwVA1zRl6VqsLs5pz5kNPXNCJeXGQEazobIkzje1OvTjwT4Q-- Received: from [98.203.21.152] by web63905.mail.re1.yahoo.com via HTTP; Mon, 14 Sep 2009 01:58:39 PDT X-Mailer: YahooMailClassic/6.1.2 YahooMailWebService/0.7.347.2 Date: Mon, 14 Sep 2009 01:58:39 -0700 (PDT) From: Barney Cordoba To: Bruce Evans , rihad In-Reply-To: <4AADF2D8.5050505@mail.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@FreeBSD.org Subject: Re: [POLLING] strange interrupt/system load X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Sep 2009 08:58:41 -0000 =0A=0A--- On Mon, 9/14/09, rihad wrote:=0A=0A> From: rihad = =0A> Subject: Re: [POLLING] strange interrupt/system load=0A= > To: "Bruce Evans" =0A> Cc: freebsd-net@FreeBSD.org= =0A> Date: Monday, September 14, 2009, 3:38 AM=0A> Bruce Evans wrote:=0A> >= On Sat, 12 Sep 2009, rihad wrote:=0A> > =0A> >> The box experiences ~230 m= bit/s traffic flow=0A> through it. I've doubled some sysctls after reading= =0A> polling(4):=0A> >> kern.polling.each_burst=3D10 # was: 5=0A> >> kern.p= olling.burst_max=3D350 # was: 150=0A> >> =0A> >> FreeBSD 7.2-RELEASE-p3 amd= 64=0A> >> HZ=3D1000=0A> > =0A> > How much better does it work without POLLI= NG?=0A> > =0A> Without polling (current load around 190-200 mbit/s, around= =0A> 24-26 kpps):=0A> =0A> top:=0A> CPU:=A0 0.0% user,=A0 0.0% nice,=A0 8.4= %=0A> system,=A0 0.0% interrupt, 91.6% idle=0A> =0A> Interrupts/s: 18322 to= tal=0A> 28 mpt0 irq16=0A> 1999 cpu0: time=0A> 6906 em0 irq256=0A> 3392 em1 = irq257=0A> 1999 cpu1: time=0A> 1999 cpu2: time=0A> 1999 cpu3: time=0A=0AYou= really need to look at the taskq usage as averaging on a 4 core=0Asystem m= uddies things up. em will generally run on 1 core per NIC, and=0Ainterrupts= are filtered so you won't see any interrupt usage. On a =0A4 core system y= ou could exhaust a core and still be at 25% overall, so =0Ayou need to watc= h the max usage per core.=0A=0AThings aren't measured properly in polling m= ode so its difficult to=0Acompare them one to one. You really don't need to= ; intuitively it makes=0Azero sense to use polling with em.=0A=0AYou'll do = a lot better setting your ITR to 2000 or so. You really don't=0Aneed an int= errupt every 4 packets at those traffic levels.=0A=0ABarney=0A=0A=0A