Date: Mon, 3 Jan 2005 14:23:25 -0800 From: "Youlin Feng" <yfeng@verniernetworks.com> To: <freebsd-arch@freebsd.org> Subject: User process starvation in FreeBSD 5.3 Message-ID: <12394E9FCB7C8441BB238D7F67B402E407E5C6@exch2.verniernetworks.com>
next in thread | raw e-mail | index | archive | help
Hello all, We are building a network appliance running FreeBSD 5.3 and under very heavy network traffic the user processes don't get scheduled for an unacceptable period of time. Marking the user process/thread real-time class doesn't help since the real-time user threads priorities are still lower than the interrupt threads. BTW, in our case, the CPU spends very long time at PI_NET (16) priority level, instead of at the (PI_SOFT + 4) soft intr level due to the packet forwarding nature. Either way, the user processes don't have a chance. In the following discussion I'll use PI_NET to represent the network interrupt threads priority. I am experimenting with improving the real-time threads scheduling performance and would like to hear from you. Here is what I am doing: 1. Raise the minimum real-time user threads priority to a value higher than PI_NET, e.g. #define PRI_MIN_REALTIME 0 And use the rtprio() syscall or command to set the priority to be higher than PI_NET, e.g. "rtprio 10 <my_proc_id>" This didn't turn out to be enough, since the real-time user process still uses system services or drivers that run at a lower priority than PI_NET, e.g. disk, tty, etc. 2. Change the PI_NET value to be the highest of all interrupt threads. Potential performance impact isn't a concern here since we have relatively rare non-network events. So PI_NET is changed to (PRI_MIN_ITHD + 32) from (PRI_MIN_ITHD + 16). This should give preference to all other I/O interrupt threads. I am assuming the software interrupt scheduling isn't the bottleneck. I haven't got a chance to reproduce the problem using the 2nd change yet. I suspect that it isn't enough to get what I want, since lots of kernel code do msleep(...,pri,...) with "pri" bigger than the changed PI_NET. But, if this thinking of changing the priority ranges makes sense, PI_NET can always be changed to the highest value of all kernel priority values, ie the lowest kernel priority. Hopefully this change would work and not be intrusive to the kernel. Thank you for your comments in advance. Youlin Feng
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?12394E9FCB7C8441BB238D7F67B402E407E5C6>