Date: Tue, 2 Apr 2002 17:49:43 -0800 (PST) From: Stefan Saroiu <tzoompy@cs.washington.edu> To: Terry Lambert <tlambert2@mindspring.com> Cc: freebsd-smp@FreeBSD.org Subject: Re: Interrupt Handlers and Multiple CPUs Message-ID: <20020402174143.B29152-100000@magnesium.cs.washington.edu> In-Reply-To: <3CAA512A.9FBEAE5@mindspring.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, 2 Apr 2002, Terry Lambert wrote:
> Stefan Saroiu wrote:
> > This is the context.
> >
> > Now the question:
> > ================
> >
> > If I drop a second CPU in the box, will FreeBSD be smart to schedule the
> > interrupts on one CPU and the packet capture code on the other? What
> > FreeBSD version do I need (I have 4.5)?
> >
> > The SMP docs on the FreeBSD site seem to point to an affirmative answer,
> > but I'd like to get more opinions before I blow more money on a second
> > CPU.
>
> [ I assume that this is for a commercial "1 Gigbit!" capture tool... ]
>
On the contrary, this is for an academic project. I'm a grad student.
> The interrupts will be scheduled on the CPU that has the vector.
>
> In the common case, this will be the same CPU that's running the
> capture application, because it's "in the kernel" waiting for I/O
> from the card, and thereore has the kernel reentrancy lock, which
> implicitly gives it the vector for the interrupts.
>
> Ideally, FreeBSD would have two other modes of operating:
>
> 1) Direct vector of interrupts, in which interrupts from
> a specific card are directed to a particular CPU, AND
>
> 2) Virtual wire mode, in which interrupts are directed
> to a "virtual" interrupt line, and whatever CPU wants
> to handle them can.
>
> The main barrier to both of these approaches is that only one
> CPU is allowed into the kernel code at a time in FreeBSD 4.x,
> and the entry barrier points are "system call", "fault handler",
> and "interrupt handler".
>
Your answer seems to (indirectly) suggest that I might have a chance with
5.0, since they do more fine grained kernel locking. Any suggestions?
> Druschel and Aron have implemented a fix for the problem you
> are seeing, called LRP ("Lazy Receiver Processing"). This has
> been available for FreeBSD since the 2.x days, and you can
> download code for FreeBSD 3.x and 4.x. The 4.x code should work
> for you. It contains accounting constructis for resource
> accounting, to ensure scheduling of user space processes, thus
> avoiding the live lock. This implementation is called ResCon
> ("Resource Containers").
>
The application still gets 20% of the CPU which is quite good actually.
Although I'm not familiar with Druschel's work, I'm not sure whether
better scheduling will help me here.
I've been toying with the idea to change the driver to raise interrupts
only once every 100 packets or something like that. Currently it is 1
interrupt per 1 packet.
> The license on this code prohibits commercial use without
> contacting the Rice University Technology Transfer office,
> but it should be enough to let you see what you are missing,
> and whether or not you want to license.
>
Again, this is purely academic, I shouldn't have a problem.
>
> In general, I would say that throwing hardware at the problem
> will not work, and any company that thinks that this is a
> workable solution, regardless of what their platform is, will
> find themselves disappointed.
>
Thanks for your answer. It's very informative.
Cheers,
--Stefan
> -- Terry
>
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-smp" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20020402174143.B29152-100000>