Date: Wed, 2 Dec 2009 06:33:58 -0800 (PST) From: Barney Cordoba <barney_cordoba@yahoo.com> To: "Yuriy A. Korobko" <administrator@shtorm.com> Cc: freebsd-net@freebsd.org Subject: Re: Reducing number of interrupts from intel pro 1000 et adapter Message-ID: <913529.24196.qm@web63906.mail.re1.yahoo.com> In-Reply-To: <1258998140.3015.34.camel@stormi-desktop>
next in thread | previous in thread | raw e-mail | index | archive | help
=0A=0A--- On Mon, 11/23/09, Yuriy A. Korobko <administrator@shtorm.com> wro= te:=0A=0A> From: Yuriy A. Korobko <administrator@shtorm.com>=0A> Subject: R= educing number of interrupts from intel pro 1000 et adapter=0A> To: freebsd= -net@freebsd.org=0A> Date: Monday, November 23, 2009, 12:42 PM=0A> Hi,=0A> = =0A> I'd like to know a way to control tx interrupts on intel=0A> pro 1000 = et=0A> adapter with igb driver. Just installed one in the router=0A> and sy= stat=0A> shows 8-9k rx interrupts and 20k tx interrupts from igb0=0A> and i= gb1=0A> adapters. Box is a router running freebsd 7.2 release, I've=0A> tri= ed=0A> default driver from kernel source and latest from intel=0A> site, ef= fect is=0A> the same with automatic interrupt moderation enabled and=0A> di= sabled. I=0A> have the same box with intel pro 1000 pt adapter which=0A> ha= ve=0A> tx(rx)_int_delay sysctls in em driver, I was able to reduce=0A> numb= er of=0A> tx/rx interrupts to 7-8k per interface and got much more=0A> cpu = idle=0A> because of less context switches with same pps.=0A> =0A> Interfaca= e load=0A> =0A> border# netstat -I igb0 -w 1=0A> =A0 =A0 =A0 =A0 =A0 =A0 in= put=A0=0A> =A0 =A0 =A0=A0=A0(igb0)=A0 =A0 =A0=0A> =A0 =A0=A0=A0output=0A> = =A0=A0=A0packets=A0 errs=A0 =A0 =A0=0A> bytes=A0 =A0 packets=A0 errs=A0 =A0= =A0=0A> bytes colls=0A> =A0 =A0=A0=A041438=A0=0A> =A0=A0=A00=A0=A0=A037923= 274=A0 =A0=0A> =A0 51173=A0=0A> =A0=A0=A00=A0=A0=A024539512=A0=0A> =A0=A0= =A00=0A> =A0 =A0=A0=A044827=A0=0A> =A0=A0=A00=A0=A0=A041626876=A0 =A0=0A> = =A0 53408=A0=0A> =A0=A0=A00=A0=A0=A024595412=A0=0A> =A0=A0=A00=0A> =A0 =A0= =A0=A043300=A0=0A> =A0=A0=A00=A0=A0=A039736056=A0 =A0=0A> =A0 53118=A0=0A> = =A0=A0=A00=A0=A0=A024574219=A0=0A> =A0=A0=A00=0A> =A0 =A0=A0=A043146=A0=0A>= =A0=A0=A00=A0=A0=A040399285=A0 =A0=0A> =A0 53455=A0=0A> =A0=A0=A00=A0=A0= =A024368290=A0=0A> =A0=A0=A00=0A> =A0 =A0=A0=A044827=A0=0A> =A0=A0=A00=A0= =A0=A042463307=A0 =A0=0A> =A0 53921=A0=0A> =A0=A0=A00=A0=A0=A023959752=A0= =0A> =A0=A0=A00=0A> =0A> Here is sysctls=0A> =0A> dev.igb.0.%desc: Intel(R)= PRO/1000 Network Connection=0A> version - 1.7.4=0A> dev.igb.0.%driver: igb= =0A> dev.igb.0.%location: slot=3D0 function=3D0=0A> dev.igb.0.%pnpinfo: ven= dor=3D0x8086 device=3D0x10c9=0A> subvendor=3D0x8086=0A> subdevice=3D0xa03c = class=3D0x020000=0A> dev.igb.0.%parent: pci1=0A> dev.igb.0.debug: -1=0A> de= v.igb.0.stats: -1=0A> dev.igb.0.flow_control: 0=0A> dev.igb.0.enable_aim: 1= =0A> dev.igb.0.low_latency: 1000=0A> dev.igb.0.ave_latency: 4000=0A> dev.ig= b.0.bulk_latency: 8000=0A> dev.igb.0.rx_processing_limit: 1000=0A> =0A> dev= .igb.1.%desc: Intel(R) PRO/1000 Network Connection=0A> version - 1.7.4=0A> = dev.igb.1.%driver: igb=0A> dev.igb.1.%location: slot=3D0 function=3D1=0A> d= ev.igb.1.%pnpinfo: vendor=3D0x8086 device=3D0x10c9=0A> subvendor=3D0x8086= =0A> subdevice=3D0xa03c class=3D0x020000=0A> dev.igb.1.%parent: pci1=0A> de= v.igb.1.debug: -1=0A> dev.igb.1.stats: -1=0A> dev.igb.1.flow_control: 0=0A>= dev.igb.1.enable_aim: 1=0A> dev.igb.1.low_latency: 1000=0A> dev.igb.1.ave_= latency: 4000=0A> dev.igb.1.bulk_latency: 8000=0A> dev.igb.1.rx_processing_= limit: 1000=0A> =0A> And debug =0A> =0A> kernel: igb0: Adapter hardware ad= dress =3D 0xc796ec1c =0A> kernel: igb0: CTRL =3D 0x40c00241 RCTL =3D 0x480= 02 =0A> kernel: igb0: Packet buffer =3D Tx=3D0k Rx=3D0k =0A> kernel: igb0= : Flow control watermarks high =3D 63488 low =3D=0A> 61988=0A> kernel: igb= 0: Queue(0) tdh =3D 3023, tdt =3D 3025=0A> kernel: igb0: TX(0) no descript= ors avail event =3D 0=0A> kernel: igb0: TX(0) MSIX IRQ Handled =3D 3754097= 484=0A> kernel: igb0: TX(0) Packets sent =3D 4815628967=0A> kernel: igb0:= Queue(0) rdh =3D 3658, rdt =3D 3645=0A> kernel: igb0: RX(0) Packets recei= ved =3D 7611879022=0A> kernel: igb0: RX(0) Split Packets =3D 0=0A> kernel= : igb0: RX(0) Byte count =3D 7013625984942=0A> kernel: igb0: RX(0) MSIX IR= Q Handled =3D 3232986641=0A> kernel: igb0: RX(0) LRO Queued=3D 0=0A> kern= el: igb0: RX(0) LRO Flushed=3D 0=0A> kernel: igb0: LINK MSIX IRQ Handled = =3D 3=0A> kernel: igb0: Mbuf defrag failed =3D 0=0A> kernel: igb0: Std mb= uf header failed =3D 0=0A> kernel: igb0: Std mbuf packet failed =3D 0=0A> = kernel: igb0: Driver dropped packets =3D 0=0A> kernel: igb0: Driver tx dm= a failure in xmit =3D 0=0A> =0A> kernel: igb1: Adapter hardware address = =3D 0xc796dc1c =0A> kernel: igb1: CTRL =3D 0x40c00241 RCTL =3D 0x48002 =0A= > kernel: igb1: Packet buffer =3D Tx=3D0k Rx=3D0k =0A> kernel: igb1: Flow= control watermarks high =3D 63488 low =3D=0A> 61988=0A> kernel: igb1: Que= ue(0) tdh =3D 4093, tdt =3D 4093=0A> kernel: igb1: TX(0) no descriptors av= ail event =3D 0=0A> kernel: igb1: TX(0) MSIX IRQ Handled =3D 10882048108= =0A> kernel: igb1: TX(0) Packets sent =3D 31169311987=0A> kernel: igb1: Q= ueue(0) rdh =3D 2515, rdt =3D 2513=0A> kernel: igb1: RX(0) Packets receive= d =3D 30747961847=0A> kernel: igb1: RX(0) Split Packets =3D 0=0A> kernel:= igb1: RX(0) Byte count =3D 26511993282060=0A> kernel: igb1: RX(0) MSIX IR= Q Handled =3D 4834518320=0A> kernel: igb1: RX(0) LRO Queued=3D 0=0A> kern= el: igb1: RX(0) LRO Flushed=3D 0=0A> kernel: igb1: LINK MSIX IRQ Handled = =3D 5=0A> kernel: igb1: Mbuf defrag failed =3D 0=0A> kernel: igb1: Std mb= uf header failed =3D 0=0A> kernel: igb1: Std mbuf packet failed =3D 0=0A> = kernel: igb1: Driver dropped packets =3D 0=0A> kernel: igb1: Driver tx dm= a failure in xmit =3D 0=0A> =0A> =0A> =0A=0AI'm curious as to why you are d= oing a load test on a single core system=0Awith a part that is clearly desi= gned to be used on a multicore system?=0A=0ABarney=0A=0A=0A
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?913529.24196.qm>