Date: Fri, 24 Jul 2009 14:10:56 -0700 (PDT) From: Barney Cordoba <barney_cordoba@yahoo.com> To: freebsd-net@FreeBSD.org, Peter Steele <psteele@webmail.maxiscale.com> Subject: Re: nfe taskq performance issues Message-ID: <624694.56110.qm@web63902.mail.re1.yahoo.com>
next in thread | raw e-mail | index | archive | help
--- On Thu, 7/23/09, Peter Steele <psteele@webmail.maxiscale.com> wrote: > From: Peter Steele <psteele@webmail.maxiscale.com> > Subject: nfe taskq performance issues > To: freebsd-net@FreeBSD.org > Date: Thursday, July 23, 2009, 11:58 AM > We've been hitting serious nfe taskq > performance issues during stress > tests and in doing some research on the problem we came > across this old > email: > > > > From: Ivan Voras <ivoras@freebsd.org> > Date: April 28, 2009 3:53:14 AM PDT > To: freebsd-threads@freebsd.org > Cc: freebsd-net@freebsd.org, > freebsd-performance@freebsd.org > Subject: Re: FreeBSD 7.1 taskq em performance > > > > I have been hitting some barrier with FreeBSD 7.1 > network performance. > I > > have written an application which contains two kernel > threads that > takes > > mbufs directly from a network interface and forwards > to another > network > > interface. This idea is to simulate different network > environment. > > > > I have been using FreeBSD 6.4 amd64 and tested with an > Ixia box > > (specialised hardware firing very high packet rate). > The PC was a > Core2 2.6 > > GHz with dual ports Intel PCIE Gigabit network card. > It can manage up > to 1.2 > > million pps. > > > > I have a higher spec PC with FreeBSD 7.1 amd64 and > Quadcore 2.3 GHz > and > > PCIE Gigabit network card. The performance can only > achieve up to 600k > pps. > > I notice the 'taskq em0' and 'taskq em1' is solid 100% > CPU but it is > not in > > FreeBSD 6.4. > > > > In our case we are running FreeBSD 7.0, but we are seeing > our boxes > experience serious thread starvation issues as the nfe0 cpu > percentage > climbs steadily while cpu idle time drops at times to 0 > percent. This > email thread mentioned a patch for the em driver here: > > > > http://people.yandex-team.ru/~wawa/ > <http://people.yandex-team.ru/%7Ewawa/>; It means you're using your CPU up processing packets. There are any number of reasons for it; lock contention, poor general design, network stack contention. I'm not sure why you'd want to use a 64 bit build for a network application, but you'll have to track down the source by profiling or running focused tests to isolate your bottlenecks. Barney
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?624694.56110.qm>