Date: Thu, 9 May 2013 09:25:20 -0700 (PDT) From: Barney Cordoba <barney_cordoba@yahoo.com> To: =?iso-8859-1?Q?Cl=E9ment_Hermann_=28nodens=29?= <nodens2099@gmail.com>, Eugene Grosbein <egrosbein@rdtc.ru> Cc: freebsd-net@freebsd.org Subject: Re: High CPU interrupt load on intel I350T4 with igb on 8.3 Message-ID: <1368116720.51479.YahooMailClassic@web121606.mail.ne1.yahoo.com> In-Reply-To: <518BB8EE.2080705@rdtc.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
--- On Thu, 5/9/13, Eugene Grosbein <egrosbein@rdtc.ru> wrote: > From: Eugene Grosbein <egrosbein@rdtc.ru> > Subject: Re: High CPU interrupt load on intel I350T4 with igb on 8.3 > To: ""Clément Hermann (nodens)"" <nodens2099@gmail.com> > Cc: freebsd-net@freebsd.org > Date: Thursday, May 9, 2013, 10:55 AM > On 26.04.2013 18:31, "Clément > Hermann (nodens)" wrote: > > Hi list, > > > > We use pf+ALTQ for trafic shaping on some routers. > > > > We are switching to new servers : Dell PowerEdge R620 > with 2 8-cores > > Intel Processor (E5-2650L), 8GB RAM and Intel I350T4 > (quad port) using > > igb driver. The old hardware is using em driver, the > CPU load is high > > but mostly due to kernel and a large pf ruleset. > > > > On the new hardware, we see high CPU Interrupt load (up > to 95%), even > > though there is not much trafic currently (peaks about > 150Mbps and > > 40Kpps). All queues are used and binded to a cpu > according to top, but a > > lot of CPU time is spent on igb queues (interrupt or > wait). The load is > > fine when we stay below 20Kpps. > > > > We see no mbuf shortage, no dropped packet, but there > is little margin > > left on CPU time (about 25% idle at best, most of CPU > time is spent on > > interrupts), which is disturbing. > > It seems you suffer from pf lock contention. You should stop > using pf > with multi-core systems with 8.3. Move to ipfw+dummynet or > ng_car for 8.3 > or move to 10.0-CURRENT having new, rewritten pf that does > not have this problem. > > Network device driver is not guilty here, that's just pf's > contention > running in igb's context. > > Eugene Grosbein They're both at play. Single threadedness aggravates subsystems that have too many lock points. It can also be "solved" with using 1 queue, because then you don't have 4 queues going into a single thread. BC
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1368116720.51479.YahooMailClassic>