Date: Thu, 9 May 2013 05:54:05 -0700 (PDT) From: Barney Cordoba <barney_cordoba@yahoo.com> To: nodens2099@gmail.com Cc: freebsd-net@freebsd.org Subject: Re: High CPU interrupt load on intel I350T4 with igb on 8.3 Message-ID: <1368104045.43950.YahooMailClassic@web121601.mail.ne1.yahoo.com> In-Reply-To: <1367175573.97340.YahooMailClassic@web121602.mail.ne1.yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
=0A=0A--- On Sun, 4/28/13, Barney Cordoba <barney_cordoba@yahoo.com> wrote:= =0A=0A> From: Barney Cordoba <barney_cordoba@yahoo.com>=0A> Subject: Re: Hi= gh CPU interrupt load on intel I350T4 with igb on 8.3=0A> To: "Jack Vogel" = <jfvogel@gmail.com>=0A> Cc: "FreeBSD Net" <freebsd-net@freebsd.org>, "Cl=E9= ment Hermann (nodens)" <nodens2099@gmail.com>=0A> Date: Sunday, April 28, 2= 013, 2:59 PM=0A> The point of lists is to be able to=0A> benefit from other= 's experiences so you don't have to waste=0A> your time "trying" things tha= t others have already done.=0A> I'm not pontificating. I've done the tests.= There's no=0A> reason for every person who is=A0having to exact same probl= em=0A> to do the same tests over and over, hoping for somemagically=0A> dif= ferent result. The result will always be the same.=0A> Because there's no c= hance=A0of it=A0working properly by=0A> chance.=0A> BC=0A> =0A> =0A> --- On= Sun, 4/28/13, Jack Vogel <jfvogel@gmail.com>=0A> wrote:=0A> =0A> From: Jac= k Vogel <jfvogel@gmail.com>=0A> Subject: Re: High CPU interrupt load on int= el I350T4 with=0A> igb on 8.3=0A> To: "Barney Cordoba" <barney_cordoba@yaho= o.com>=0A> Cc: "FreeBSD Net" <freebsd-net@freebsd.org>,=0A> "Cl=E9ment Herm= ann (nodens)" <nodens2099@gmail.com>=0A> Date: Sunday, April 28, 2013, 1:07= PM=0A> =0A> Try setting your queues to 1, run some tests, then try=0A> set= tingyour queues to 2, then to 4... its called tuning, and=0A> rather thanju= st pontificating about it, which Barney so=0A> loves to do, you can=0A> dis= cover what works best. I ran tests last week preparing=0A> for anew driver = version and found the best results came not=0A> only whiletweaking queues, = but also ring size, and I could=0A> see changes based=0A> on the buf ring s= ize.... =A0There are lots of things that may=0A> improve ordegrade performa= nce depending on the workload.=0A> Jack=0A> =0A> =0A> =0A> On Sun, Apr 28, = 2013 at 7:21 AM, Barney Cordoba <barney_cordoba@yahoo.com>=0A> wrote:=0A> = =0A> =0A> =0A> =0A> =0A> --- On Fri, 4/26/13, "Cl=E9ment Hermann (nodens)" = <nodens2099@gmail.com>=0A> wrote:=0A> =0A> =0A> =0A> > From: "Cl=E9ment Her= mann (nodens)" <nodens2099@gmail.com>=0A> =0A> > Subject: High CPU interrup= t load on intel I350T4 with=0A> igb on 8.3=0A> =0A> > To: freebsd-net@freeb= sd.org=0A> =0A> > Date: Friday, April 26, 2013, 7:31 AM=0A> =0A> > Hi list,= =0A> =0A> >=0A> =0A> > We use pf+ALTQ for trafic shaping on some routers.= =0A> =0A> >%> =0A> > We are switching to new servers : Dell PowerEdge R620= =0A> with 2=0A> =0A> > 8-cores Intel Processor (E5-2650L), 8GB RAM and Inte= l=0A> I350T4=0A> =0A> > (quad port) using igb driver. The old hardware is u= sing=0A> em=0A> =0A> > driver, the CPU load is high but mostly due to kerne= l=0A> and a=0A> =0A> > large pf ruleset.=0A> =0A> >=0A> =0A> > On the new h= ardware, we see high CPU Interrupt load (up=0A> to=0A> =0A> > 95%), even th= ough there is not much trafic currently=0A> (peaks=0A> =0A> > about 150Mbps= and 40Kpps). All queues are used and=0A> binded to=0A> =0A> > a cpu accord= ing to top, but a lot of CPU time is spent=0A> on=0A> =0A> > igb queues (in= terrupt or wait). The load is fine when=0A> we=0A> =0A> > stay below 20Kpps= .=0A> =0A> >=0A> =0A> > We see no mbuf shortage, no dropped packet, but the= re=0A> is=0A> =0A> > little margin left on CPU time (about 25% idle at best= ,=0A> most=0A> =0A> > of CPU time is spent on interrupts), which is=0A> dis= turbing.=0A> =0A> >=0A> =0A> > We have done some tuning, but to no avail := =0A> =0A> >=0A> =0A> > sysctl.conf :=0A> =0A> >=0A> =0A> > # mbufs=0A> =0A>= > kern.ipc.nmbclusters=3D65536=0A> =0A> > # Sockets=0A> =0A> > kern.ipc.so= maxconn=3D8192=0A> =0A> > net.inet.tcp.delayed_ack=3D0=0A> =0A> > net.inet.= tcp.sendspace=3D65535=0A> =0A> > net.inet.udp.recvspace=3D65535=0A> =0A> > = net.inet.udp.maxdgram=3D57344=0A> =0A> > net.local.stream.recvspace=3D65535= =0A> =0A> > net.local.stream.sendspace=3D65535=0A> =0A> > # IGB=0A> =0A> > = dev.igb.0.rx_processing_limit=3D4096=0A> =0A> > dev.igb.1.rx_processing_lim= it=3D4096=0A> =0A> > dev.igb.2.rx_processing_limit=3D4096=0A> =0A> > dev.ig= b.3.rx_processing_limit=3D4096=0A> =0A> >=0A> =0A> > /boot/loader.conf :=0A= > =0A> >=0A> =0A> > vm.kmem_size=3D1G=0A> =0A> > hw.igb.max_interrupt_rate= =3D"32000"=A0 # maximum number=0A> of=0A> =0A> > interrupts/sec generated b= y single igb(4) (default=0A> 8000)=0A> =0A> > hw.igb.txd=3D"2048"=A0 =A0 = =A0 =A0 =A0 =A0=0A> =0A> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=0A> =0A> > =A0 #= number of transmit descriptors allocated by the=0A> =0A> > driver (2048 li= mit)=0A> =0A> > hw.igb.rxd=3D"2048"=A0 =A0 =A0 =A0 =A0 =A0=0A> =0A> > =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0=0A> =0A> > =A0 # number of receive descriptors = allocated by the=0A> =0A> > driver (2048 limit)=0A> =0A> > hw.igb.rx_proces= s_limit=3D"1000"=A0 =A0=A0=A0#=0A> =0A> > maximum number of received packet= s to process at a=0A> time, The=0A> =0A> > default of 100 is=0A> =0A> > =A0= =A0 =A0 =A0 =A0 =A0 =A0 =A0=0A> =0A> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=0A>= =0A> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=0A> =0A> > =A0 =A0 =A0 =A0 =A0 =A0= =0A> =0A> > =A0=A0=A0# too low for most firewalls. (-1 means=0A> =0A> > unl= imited)=0A> =0A> >=0A> =0A> > Kernel HZ is 1000.=0A> =0A> >=0A> =0A> > The = IGB /boot/loader.conf tuning was our last attempt,=0A> it=0A> =0A> > didn't= change anything.=0A> =0A> >=0A> =0A> > Does anyone have any pointer ? How = could we lower CPU=0A> =0A> > interrupt load ? should we set=0A> hw.igb.max= _interrupt_rate=0A> =0A> > lower instead of higher ?=0A> =0A> > From what w= e saw here and there, we should be able to=0A> do=0A> =0A> > much better wi= th this hardware.=0A> =0A> >=0A> =0A> >=0A> =0A> > relevant sysctl (igb1 an= d igb2 only, other interfaces=0A> are=0A> =0A> > unused) :=0A> =0A> >=0A> = =0A> > sysctl dev.igb | grep -v ": 0$"=0A> =0A> > dev.igb.1.%desc: Intel(R)= PRO/1000 Network Connection=0A> =0A> > version - 2.3.1=0A> =0A> > dev.igb.= 1.%driver: igb=0A> =0A> > dev.igb.1.%location: slot=3D0 function=3D1=0A> = =0A> > dev.igb.1.%pnpinfo: vendor=3D0x8086 device=3D0x1521=0A> =0A> > subve= ndor=3D0x8086 subdevice=3D0x5001 class=3D0x020000=0A> =0A> > dev.igb.1.%par= ent: pci5=0A> =0A> > dev.igb.1.nvm: -1=0A> =0A> > dev.igb.1.enable_aim: 1= =0A> =0A> > dev.igb.1.fc: 3=0A> =0A> > dev.igb.1.rx_processing_limit: 4096= =0A> =0A> > dev.igb.1.eee_disabled: 1=0A> =0A> > dev.igb.1.link_irq: 2=0A> = =0A> > dev.igb.1.device_control: 1209795137=0A> =0A> > dev.igb.1.rx_control= : 67141658=0A> =0A> > dev.igb.1.interrupt_mask: 4=0A> =0A> > dev.igb.1.exte= nded_int_mask: 2147483981=0A> =0A> > dev.igb.1.fc_high_water: 33168=0A> =0A= > > dev.igb.1.fc_low_water: 33152=0A> =0A> > dev.igb.1.queue0.interrupt_rat= e: 71428=0A> =0A> > dev.igb.1.queue0.txd_head: 1318=0A> =0A> > dev.igb.1.qu= eue0.txd_tail: 1318=0A> =0A> > dev.igb.1.queue0.tx_packets: 84663594=0A> = =0A> > dev.igb.1.queue0.rxd_head: 717=0A> =0A> > dev.igb.1.queue0.rxd_tail:= 715=0A> =0A> > dev.igb.1.queue0.rx_packets: 43899597=0A> =0A> > dev.igb.1.= queue0.rx_bytes: 8905556030=0A> =0A> > dev.igb.1.queue1.interrupt_rate: 909= 09=0A> =0A> > dev.igb.1.queue1.txd_head: 693=0A> =0A> > dev.igb.1.queue1.tx= d_tail: 693=0A> =0A> > dev.igb.1.queue1.tx_packets: 57543349=0A> =0A> > dev= .igb.1.queue1.rxd_head: 1033=0A> =0A> > dev.igb.1.queue1.rxd_tail: 1032=0A>= =0A> > dev.igb.1.queue1.rx_packets: 54821897=0A> =0A> > dev.igb.1.queue1.r= x_bytes: 9944955108=0A> =0A> > dev.igb.1.queue2.interrupt_rate: 100000=0A> = =0A> > dev.igb.1.queue2.txd_head: 350=0A> =0A> > dev.igb.1.queue2.txd_tail:= 350=0A> =0A> > dev.igb.1.queue2.tx_packets: 62320990=0A> =0A> > dev.igb.1.= queue2.rxd_head: 1962=0A> =0A> > dev.igb.1.queue2.rxd_tail: 1939=0A> =0A> >= dev.igb.1.queue2.rx_packets: 43909016=0A> =0A> > dev.igb.1.queue2.rx_bytes= : 8673941461=0A> =0A> > dev.igb.1.queue3.interrupt_rate: 14925=0A> =0A> > d= ev.igb.1.queue3.txd_head: 647=0A> =0A> > dev.igb.1.queue3.txd_tail: 647=0A>= =0A> > dev.igb.1.queue3.tx_packets: 58776199=0A> =0A> > dev.igb.1.queue3.r= xd_head: 692=0A> =0A> > dev.igb.1.queue3.rxd_tail: 691=0A> =0A> > dev.igb.1= .queue3.rx_packets: 55138996=0A> =0A> > dev.igb.1.queue3.rx_bytes: 93102173= 54=0A> =0A> > dev.igb.1.queue4.interrupt_rate: 100000=0A> =0A> > dev.igb.1.= queue4.txd_head: 1721=0A> =0A> > dev.igb.1.queue4.txd_tail: 1721=0A> =0A> >= dev.igb.1.queue4.tx_packets: 54337209=0A> =0A> > dev.igb.1.queue4.rxd_head= : 1609=0A> =0A> > dev.igb.1.queue4.rxd_tail: 1598=0A> =0A> > dev.igb.1.queu= e4.rx_packets: 46546503=0A> =0A> > dev.igb.1.queue4.rx_bytes: 8818182840=0A= > =0A> > dev.igb.1.queue5.interrupt_rate: 11627=0A> =0A> > dev.igb.1.queue5= .txd_head: 254=0A> =0A> > dev.igb.1.queue5.txd_tail: 254=0A> =0A> > dev.igb= .1.queue5.tx_packets: 53117182=0A> =0A> > dev.igb.1.queue5.rxd_head: 701=0A= > =0A> > dev.igb.1.queue5.rxd_tail: 685=0A> =0A> > dev.igb.1.queue5.rx_pack= ets: 43014837=0A> =0A> > dev.igb.1.queue5.rx_bytes: 8699057447=0A> =0A> > d= ev.igb.1.queue6.interrupt_rate: 55555=0A> =0A> > dev.igb.1.queue6.txd_head:= 8=0A> =0A> > dev.igb.1.queue6.txd_tail: 8=0A> =0A> > dev.igb.1.queue6.tx_p= ackets: 52654088=0A> =0A> > dev.igb.1.queue6.rxd_head: 1057=0A> =0A> > dev.= igb.1.queue6.rxd_tail: 1041=0A> =0A> > dev.igb.1.queue6.rx_packets: 4522703= 0=0A> =0A> > dev.igb.1.queue6.rx_bytes: 9494489640=0A> =0A> > dev.igb.1.que= ue7.interrupt_rate: 5235=0A> =0A> > dev.igb.1.queue7.txd_head: 729=0A> =0A>= > dev.igb.1.queue7.txd_tail: 729=0A> =0A> > dev.igb.1.queue7.tx_packets: 6= 1926105=0A> =0A> > dev.igb.1.queue7.rxd_head: 146=0A> =0A> > dev.igb.1.queu= e7.rxd_tail: 140=0A> =0A> > dev.igb.1.queue7.rx_packets: 51781775=0A> =0A> = > dev.igb.1.queue7.rx_bytes: 8901279226=0A> =0A> > dev.igb.1.mac_stats.miss= ed_packets: 1657=0A> =0A> > dev.igb.1.mac_stats.recv_no_buff: 405=0A> =0A> = > dev.igb.1.mac_stats.total_pkts_recvd: 384332760=0A> =0A> > dev.igb.1.mac_= stats.good_pkts_recvd: 384331103=0A> =0A> > dev.igb.1.mac_stats.bcast_pkts_= recvd: 15510=0A> =0A> > dev.igb.1.mac_stats.mcast_pkts_recvd: 52957=0A> =0A= > > dev.igb.1.mac_stats.rx_frames_64: 195496498=0A> =0A> > dev.igb.1.mac_st= ats.rx_frames_65_127: 133346124=0A> =0A> > dev.igb.1.mac_stats.rx_frames_12= 8_255: 5254911=0A> =0A> > dev.igb.1.mac_stats.rx_frames_256_511: 9700049=0A= > =0A> > dev.igb.1.mac_stats.rx_frames_512_1023: 16885886=0A> =0A> > dev.ig= b.1.mac_stats.rx_frames_1024_1522: 23647635=0A> =0A> > dev.igb.1.mac_stats.= good_octets_recvd: 74284029276=0A> =0A> > dev.igb.1.mac_stats.good_octets_t= xd: 544536708502=0A> =0A> > dev.igb.1.mac_stats.total_pkts_txd: 485327419= =0A> =0A> > dev.igb.1.mac_stats.good_pkts_txd: 485327419=0A> =0A> > dev.igb= .1.mac_stats.bcast_pkts_txd: 72=0A> =0A> > dev.igb.1.mac_stats.mcast_pkts_t= xd: 52820=0A> =0A> > dev.igb.1.mac_stats.tx_frames_64: 57820809=0A> =0A> > = dev.igb.1.mac_stats.tx_frames_65_127: 51586341=0A> =0A> > dev.igb.1.mac_sta= ts.tx_frames_128_255: 7050579=0A> =0A> > dev.igb.1.mac_stats.tx_frames_256_= 511: 7887126=0A> =0A> > dev.igb.1.mac_stats.tx_frames_512_1023: 10130891=0A= > =0A> > dev.igb.1.mac_stats.tx_frames_1024_1522: 350851673=0A> =0A> > dev.= igb.1.interrupts.asserts: 551135045=0A> =0A> > dev.igb.1.interrupts.rx_pkt_= timer: 384326679=0A> =0A> > dev.igb.1.interrupts.tx_queue_empty: 485323376= =0A> =0A> > dev.igb.1.interrupts.tx_queue_min_thresh: 6324386=0A> =0A> > de= v.igb.1.host.rx_pkt: 4424=0A> =0A> > dev.igb.1.host.tx_good_pkt: 4043=0A> = =0A> > dev.igb.1.host.rx_good_bytes: 74284030864=0A> =0A> > dev.igb.1.host.= tx_good_bytes: 544536708502=0A> =0A> > dev.igb.2.%desc: Intel(R) PRO/1000 N= etwork Connection=0A> =0A> > version - 2.3.1=0A> =0A> > dev.igb.2.%driver: = igb=0A> =0A> > dev.igb.2.%location: slot=3D0 function=3D2=0A> =0A> > dev.ig= b.2.%pnpinfo: vendor=3D0x8086 device=3D0x1521=0A> =0A> > subvendor=3D0x8086= subdevice=3D0x5001 class=3D0x020000=0A> =0A> > dev.igb.2.%parent: pci5=0A>= =0A> > dev.igb.2.nvm: -1=0A> =0A> > dev.igb.2.enable_aim: 1=0A> =0A> > dev= .igb.2.fc: 3=0A> =0A> > dev.igb.2.rx_processing_limit: 4096=0A> =0A> > dev.= igb.2.eee_disabled: 1=0A> =0A> > dev.igb.2.link_irq: 2=0A> =0A> > dev.igb.2= .device_control: 1209795137=0A> =0A> > dev.igb.2.rx_control: 67141658=0A> = =0A> > dev.igb.2.interrupt_mask: 4=0A> =0A> > dev.igb.2.extended_int_mask: = 2147483959=0A> =0A> > dev.igb.2.fc_high_water: 33168=0A> =0A> > dev.igb.2.f= c_low_water: 33152=0A> =0A> > dev.igb.2.queue0.interrupt_rate: 13698=0A> = =0A> > dev.igb.2.queue0.txd_head: 1618=0A> =0A> > dev.igb.2.queue0.txd_tail= : 1618=0A> =0A> > dev.igb.2.queue0.tx_packets: 46401106=0A> =0A> > dev.igb.= 2.queue0.rxd_head: 831=0A> =0A> > dev.igb.2.queue0.rxd_tail: 827=0A> =0A> >= dev.igb.2.queue0.rx_packets: 69356350=0A> =0A> > dev.igb.2.queue0.rx_bytes= : 68488772907=0A> =0A> > dev.igb.2.queue1.interrupt_rate: 5405=0A> =0A> > d= ev.igb.2.queue1.txd_head: 190=0A> =0A> > dev.igb.2.queue1.txd_tail: 190=0A>= =0A> > dev.igb.2.queue1.tx_packets: 55965886=0A> =0A> > dev.igb.2.queue1.r= xd_head: 268=0A> =0A> > dev.igb.2.queue1.rxd_tail: 256=0A> =0A> > dev.igb.2= .queue1.rx_packets: 58958084=0A> =0A> > dev.igb.2.queue1.rx_bytes: 69154569= 937=0A> =0A> > dev.igb.2.queue2.interrupt_rate: 83333=0A> =0A> > dev.igb.2.= queue2.txd_head: 568=0A> =0A> > dev.igb.2.queue2.txd_tail: 568=0A> =0A> > d= ev.igb.2.queue2.tx_packets: 44974648=0A> =0A> > dev.igb.2.queue2.rxd_head: = 371=0A> =0A> > dev.igb.2.queue2.rxd_tail: 219=0A> =0A> > dev.igb.2.queue2.r= x_packets: 67037407=0A> =0A> > dev.igb.2.queue2.rx_bytes: 72042326102=0A> = =0A> > dev.igb.2.queue3.interrupt_rate: 12658=0A> =0A> > dev.igb.2.queue3.t= xd_head: 867=0A> =0A> > dev.igb.2.queue3.txd_tail: 867=0A> =0A> > dev.igb.2= .queue3.tx_packets: 55962467=0A> =0A> > dev.igb.2.queue3.rxd_head: 85=0A> = =0A> > dev.igb.2.queue3.rxd_tail: 1953=0A> =0A> > dev.igb.2.queue3.rx_packe= ts: 60972965=0A> =0A> > dev.igb.2.queue3.rx_bytes: 70397176035=0A> =0A> > d= ev.igb.2.queue4.interrupt_rate: 90909=0A> =0A> > dev.igb.2.queue4.txd_head:= 1920=0A> =0A> > dev.igb.2.queue4.txd_tail: 1920=0A> =0A> > dev.igb.2.queue= 4.tx_packets: 47660931=0A> =0A> > dev.igb.2.queue4.rxd_head: 1397=0A> =0A> = > dev.igb.2.queue4.rxd_tail: 1379=0A> =0A> > dev.igb.2.queue4.rx_packets: 5= 9110758=0A> =0A> > dev.igb.2.queue4.rx_bytes: 68919201478=0A> =0A> > dev.ig= b.2.queue5.interrupt_rate: 111111=0A> =0A> > dev.igb.2.queue5.txd_head: 886= =0A> =0A> > dev.igb.2.queue5.txd_tail: 886=0A> =0A> > dev.igb.2.queue5.tx_p= ackets: 45103990=0A> =0A> > dev.igb.2.queue5.rxd_head: 812=0A> =0A> > dev.i= gb.2.queue5.rxd_tail: 799=0A> =0A> > dev.igb.2.queue5.rx_packets: 59030312= =0A> =0A> > dev.igb.2.queue5.rx_bytes: 69234293962=0A> =0A> > dev.igb.2.que= ue6.interrupt_rate: 5208=0A> =0A> > dev.igb.2.queue6.txd_head: 1926=0A> =0A= > > dev.igb.2.queue6.txd_tail: 1926=0A> =0A> > dev.igb.2.queue6.tx_packets:= 46215046=0A> =0A> > dev.igb.2.queue6.rxd_head: 692=0A> =0A> > dev.igb.2.qu= eue6.rxd_tail: 689=0A> =0A> > dev.igb.2.queue6.rx_packets: 58256050=0A> =0A= > > dev.igb.2.queue6.rx_bytes: 68429172749=0A> =0A> > dev.igb.2.queue7.inte= rrupt_rate: 50000=0A> =0A> > dev.igb.2.queue7.txd_head: 126=0A> =0A> > dev.= igb.2.queue7.txd_tail: 126=0A> =0A> > dev.igb.2.queue7.tx_packets: 52451455= =0A> =0A> > dev.igb.2.queue7.rxd_head: 968=0A> =0A> > dev.igb.2.queue7.rxd_= tail: 885=0A> =0A> > dev.igb.2.queue7.rx_packets: 65946491=0A> =0A> > dev.i= gb.2.queue7.rx_bytes: 70263478849=0A> =0A> > dev.igb.2.mac_stats.missed_pac= kets: 958=0A> =0A> > dev.igb.2.mac_stats.recv_no_buff: 69=0A> =0A> > dev.ig= b.2.mac_stats.total_pkts_recvd: 498658079=0A> =0A> > dev.igb.2.mac_stats.go= od_pkts_recvd: 498657121=0A> =0A> > dev.igb.2.mac_stats.bcast_pkts_recvd: 1= 6867=0A> =0A> > dev.igb.2.mac_stats.mcast_pkts_recvd: 52957=0A> =0A> > dev.= igb.2.mac_stats.rx_frames_64: 59089332=0A> =0A> > dev.igb.2.mac_stats.rx_fr= ames_65_127: 52880118=0A> =0A> > dev.igb.2.mac_stats.rx_frames_128_255: 752= 6966=0A> =0A> > dev.igb.2.mac_stats.rx_frames_256_511: 8468389=0A> =0A> > d= ev.igb.2.mac_stats.rx_frames_512_1023: 10434770=0A> =0A> > dev.igb.2.mac_st= ats.rx_frames_1024_1522: 360257545=0A> =0A> > dev.igb.2.mac_stats.good_octe= ts_recvd: 558910494322=0A> =0A> > dev.igb.2.mac_stats.good_octets_txd: 8461= 8858153=0A> =0A> > dev.igb.2.mac_stats.total_pkts_txd: 394726904=0A> =0A> >= dev.igb.2.mac_stats.good_pkts_txd: 394726904=0A> =0A> > dev.igb.2.mac_stat= s.bcast_pkts_txd: 48=0A> =0A> > dev.igb.2.mac_stats.mcast_pkts_txd: 52821= =0A> =0A> > dev.igb.2.mac_stats.tx_frames_64: 196605932=0A> =0A> > dev.igb.= 2.mac_stats.tx_frames_65_127: 134602807=0A> =0A> > dev.igb.2.mac_stats.tx_f= rames_128_255: 5705236=0A> =0A> > dev.igb.2.mac_stats.tx_frames_256_511: 10= 267168=0A> =0A> > dev.igb.2.mac_stats.tx_frames_512_1023: 17165496=0A> =0A>= > dev.igb.2.mac_stats.tx_frames_1024_1522: 30380265=0A> =0A> > dev.igb.2.i= nterrupts.asserts: 465994260=0A> =0A> > dev.igb.2.interrupts.rx_pkt_timer: = 498647546=0A> =0A> > dev.igb.2.interrupts.tx_queue_empty: 394720657=0A> =0A= > > dev.igb.2.interrupts.tx_queue_min_thresh: 24424555=0A> =0A> > dev.igb.2= .host.rx_pkt: 9575=0A> =0A> > dev.igb.2.host.tx_good_pkt: 6248=0A> =0A> > d= ev.igb.2.host.rx_good_bytes: 558910513984=0A> =0A> > dev.igb.2.host.tx_good= _bytes: 84618858217=0A> =0A> >=0A> =0A> >=0A> =0A> > Thanks for your help.= =0A> =0A> >=0A> =0A> > Cheers,=0A> =0A> =0A> =0A> You're experiencing lock = contention=0A> =0A> =0A> =0A> Try editing igb.c and setting the queues to 1= . The=0A> multiqueue=0A> =0A> implementation in igb has a negative impact.= =0A> =0A> =0A> =0A> If you have just 1 system get a dual port 82571 card an= d use=0A> the=0A> =0A> em driver.=0A> =0A> =0A> =0A> BC=0A> =0A=0AI'm curio= us as to why you are going to spend $4000. on hardware rather=0Athan just b= uying a commercial firewall? You shouldn't need 16 cores to=0Afilter 150Kpp= s. You can do that with 2 cores.=0A=0ABC
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1368104045.43950.YahooMailClassic>