Date: Mon, 3 Jan 2005 14:23:25 -0800 From: "Youlin Feng" <yfeng@verniernetworks.com> To: <freebsd-arch@freebsd.org> Subject: User process starvation in FreeBSD 5.3 Message-ID: <12394E9FCB7C8441BB238D7F67B402E407E5C6@exch2.verniernetworks.com>
next in thread | raw e-mail | index | archive | help
Hello all, =20 We are building a network appliance running FreeBSD 5.3 and under very heavy network traffic the user processes don't get scheduled for an unacceptable period of time. Marking the user process/thread real-time class doesn't help since the real-time user threads priorities are still lower than the interrupt threads. BTW, in our case, the CPU spends very long time at PI_NET (16) priority level, instead of at the (PI_SOFT + 4) soft intr level due to the packet forwarding nature. Either way, the user processes don't have a chance. In the following discussion I'll use PI_NET to represent the network interrupt threads priority. =20 I am experimenting with improving the real-time threads scheduling performance and would like to hear from you. =20 Here is what I am doing: 1. Raise the minimum real-time user threads priority to a value higher than PI_NET, e.g.=20 #define PRI_MIN_REALTIME 0 And use the rtprio() syscall or command to set the priority to be higher than PI_NET, e.g. "rtprio 10 <my_proc_id>" =20 This didn't turn out to be enough, since the real-time user process still uses system services or drivers that run at a lower priority than PI_NET, e.g. disk, tty, etc. =20 2. Change the PI_NET value to be the highest of all interrupt threads. Potential performance impact isn't a concern here since we have relatively rare non-network events. So PI_NET is changed to (PRI_MIN_ITHD + 32) from (PRI_MIN_ITHD + 16). This should give preference to all other I/O interrupt threads. I am assuming the software interrupt scheduling isn't the bottleneck. =20 I haven't got a chance to reproduce the problem using the 2nd change yet. I suspect that it isn't enough to get what I want, since lots of kernel code do msleep(...,pri,...) with "pri" bigger than the changed PI_NET. But, if this thinking of changing the priority ranges makes sense, PI_NET can always be changed to the highest value of all kernel priority values, ie the lowest kernel priority. =20 Hopefully this change would work and not be intrusive to the kernel. =20 Thank you for your comments in advance. =20 Youlin Feng
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?12394E9FCB7C8441BB238D7F67B402E407E5C6>