Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 3 Jan 2005 17:43:16 -0800
From:      "Youlin Feng" <yfeng@verniernetworks.com>
To:        <freebsd-net@freebsd.org>
Subject:   User process starvation under heavy network traffic  in FreeBSD 5.3
Message-ID:  <12394E9FCB7C8441BB238D7F67B402E407E5E1@exch2.verniernetworks.com>

next in thread | raw e-mail | index | archive | help
Hello,

=20

I posted this question to freebsd-arch earlier. Stefan Bethke suggested
that I try freebsd-net instead.

=20

We are building a network appliance running FreeBSD 5.3 and under very
heavy network traffic the user processes don't get scheduled for an
unacceptable period of time. Marking the user process/thread real-time
class doesn't help since the real-time user threads priorities are still
lower than the interrupt threads. BTW, in our case, the CPU spends very
long time at PI_NET (16) priority level, instead of at the (PI_SOFT + 4)
soft intr level due to the packet forwarding nature. Either way, the
user processes don't have a chance. In the following discussion I'll use
PI_NET to represent the network interrupt threads priority.

=20

I am experimenting with improving the real-time threads scheduling
performance and would like to hear from you.

=20

Here is what I am doing:

1.	Raise the minimum real-time user threads priority to a value
higher than PI_NET, e.g.=20

#define PRI_MIN_REALTIME            0

And use the rtprio() syscall or command to set the priority to be higher
than PI_NET, e.g. "rtprio 10 <my_proc_id>"

=20

This didn't turn out to be enough, since the real-time user process
still uses system services or drivers that run at a lower priority than
PI_NET, e.g. disk, tty, etc.

=20

2.	Change the PI_NET value to be the highest of all interrupt
threads. Potential performance impact isn't a concern here since we have
relatively rare non-network events. So PI_NET is changed to
(PRI_MIN_ITHD + 32) from (PRI_MIN_ITHD + 16). This should give
preference to all other I/O interrupt threads. I am assuming the
software interrupt scheduling isn't the bottleneck.=20

=20

I haven't got a chance to reproduce the problem using the 2nd change
yet. I suspect that it isn't enough to get what I want, since lots of
kernel code do msleep(...,pri,...) with "pri" bigger than the changed
PI_NET. But, if this thinking of changing the priority ranges makes
sense, PI_NET can always be changed to the highest value of all kernel
priority values, ie the lowest kernel priority.

=20

Hopefully this change would work and not be intrusive to the kernel.

=20

Thank you for your comments in advance.

=20

Youlin Feng

=20



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?12394E9FCB7C8441BB238D7F67B402E407E5E1>