Date: Thu, 12 Mar 2009 17:18:37 -0700 (PDT) From: Barney Cordoba <barney_cordoba@yahoo.com> To: Scott Long <scottl@samsco.org> Cc: current@freebsd.org Subject: Re: Interrupt routine usage not shown by top in 8.0 Message-ID: <857006.73926.qm@web63904.mail.re1.yahoo.com> In-Reply-To: <49B99DE3.6050401@samsco.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--- On Thu, 3/12/09, Scott Long <scottl@samsco.org> wrote:
> From: Scott Long <scottl@samsco.org>
> Subject: Re: Interrupt routine usage not shown by top in 8.0
> To: barney_cordoba@yahoo.com
> Cc: current@freebsd.org
> Date: Thursday, March 12, 2009, 7:42 PM
> Barney Cordoba wrote:
> > I'm fireing 400Kpps at a udp blackhole port.
> I'm getting 6000 interrupts
> > per second on em3:
> >
> > testbox# vmstat -i; sleep 1; vmstat -i
> > interrupt total rate
> > irq1: atkbd0 1 0
> > irq6: fdc0 1 0
> > irq17: uhci1+ 2226 9
> > irq18: uhci2 ehci+ 9 0
> > cpu0: timer 470507 1993
> > irq256: em0 665 2
> > irq259: em3 1027684 4354
> > cpu1: timer 470272 1992
> > cpu3: timer 470273 1992
> > cpu2: timer 470273 1992
> > Total 2911911 12338
> >
> > interrupt total rate
> > irq1: atkbd0 1 0
> > irq6: fdc0 1 0
> > irq17: uhci1+ 2226 9
> > irq18: uhci2 ehci+ 9 0
> > cpu0: timer 472513 1993
> > irq256: em0 668 2
> > irq259: em3 1033703 4361
> > cpu1: timer 472278 1992
> > cpu3: timer 472279 1992
> > cpu2: timer 472279 1992
> > Total 2925957 12345
> >
> >
> > top -SH shows:
> >
> > PID STATE C TIME CPU COMMAND
> > 10 CPU3 3 7:32 100.00% idle
> > 10 CPU2 2 7:32 100.00% idle
> > 10 RUN 0 7:31 100.00% idle
> > 10 CPU1 1 7:31 100.00% idle
> >
> > This implies that CPU usage is substantially
> under-reported in general
> > by the system. Note that I've modified
> em_irq_fast() to call em_handle_rxtx() directly rather than
> scheduling a task to illustrate
> > the problem
> >
>
> With unmodified code, what do you see? Are you sending
> valid UDP frames with valid checksums and a valid port, or
> is everything that you're blasting at the interface
> getting dropped right away? Calling em_handle_rxtx()
> directly will cause a very quick panic once you start
> handling real traffic and you encounter a lock.
>
> Scott
I think you're mistaken. I'm also accessing the system via an em port
(and running top) and em_handle_rxtx() is self contained lock-wise.
The taskqueue doesn't obtain a lock before calling the routine.
As I mentioned, they're being dumped into a udp blackhole, which implies
that I have udp.blackhole set and the port is unused. I can see the
packets hit the udp socket so its working as expected:
853967872 dropped due to no socket
With unmodified code, the tasq shows 25% usage or so.
I'm not sure what the point of your criticism for what clearly is a test.
Are you implying that the system can receive 400K pps with 6000 ints/sec
and record 0% usage because of a coding imperfection? Or are you implying
that the 25% usage is all due to launching tasks unnecessarily and process
switching?
Barney
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?857006.73926.qm>