Date: Wed, 08 Oct 2003 00:39:31 -0700 From: Terry Lambert <tlambert2@mindspring.com> To: "Giovanni P. Tirloni" <gpt@tirloni.org> Cc: freebsd-hackers@freebsd.org Subject: Re: netisr Message-ID: <3F83BF33.46696D6C@mindspring.com> References: <20031007144635.GP25542@pixies.tirloni.org>
next in thread | previous in thread | raw e-mail | index | archive | help
"Giovanni P. Tirloni" wrote: > I'm studying the network stack and now I'm confronted with something > called netisr. It seems ether_demux puts the packet in a netisr queue > instead of passing it directly to ip_input (if that was the packet's > type). Is this derived from LRP ? No. NETISR is a software interrupt that runs when software interrupts run, which is to say, when the SPL is lowered as a result of returning from the last nested hardware interrupt, which means on hardware and clock interrupts. It is completely antithetical to LRP, which, despite the name "Lazy Receiver Processing", is mostly only lazy about direct dispatch. > I've read their paper and it looks > like their network channel (ni) but I'm not sure. No. There are two LRP papers from Rice University. The first was against FreeBSD 3.2, and dealt with just the idea of LRP. The scond makes things much more complicated than they need to be, in order to introduce a concept called "ResCon", or resource containers. Neither set of code is really production, because it uses an alternate protocol family definition and a seperately hacked uipc_socket.c and uipc_socket2.c, along with the totally parallel TCP/IP stack. The closest you can come to LRP in FreeBSD 5.x is to use the sysctl's for "direct dispatch", which will, in fact, directly call ip_input() from interrupt. This isn't a full LRP, since it doesn't add a receive buffer into the proc structure for per-process enqueuing of packets. When I implemented the 3.x version of Rice's LRP in FreeBSD 4.3, I avoided this hack. The main reason for the hack was to deal with accepting connections, since at interrupt, without a proc structure, there was no way to deal with the socket creation for the accept, due to a lack of an appropriate credential. The sneaky approach I used for this was to create the accept socket using the cred that was present on the listen socket on which the connection had come in. For this to be at all useful, you need to extend kevent accept filters to allow creation of accepted descriptor instances in the process context, and throw them onto the kqueue that was set up against the listen socket. I recommend that if you want to play with LRP, you add an attribute flag to the protocol stack structure to indicate an LRP capable vs. an LRP incapable stack, and then implement it inline, rather than as a separate thing. I also recommend that if you do this, you do it using the Rice 3.x code, and ignore the "ResCon" stuff, which I think is an interesting idea, but which adds very little in the way of real value to things (though it does add overhead). > Where can I find more information on this? If you are asking about NETISR, then I recommend W. Richard Steven's books, specifically "TCP/IP Illustrated, Volume 2: The Implementation". If you are asking about LRP, then any search engine search for "Lazy Receiver Processing" should turn up the two Rice University and the one Duke University reference, as well as dozens of other references, including the one to the Lucent/IBM work on QLinux (which has some other neat information on things like WFS Queueing and other things that are actually necessary to avoid the potential for livelock at the user/kernel boundary). FWIW: Interrupt threads, as they are used in 5.x, are pretty much antithetical to an LRP implementation, since you can still end up live-locked under heavy load (or denial of service attack), which is why you wanted LRP in the first place: to make progress under a load heavier than you could possibly respond to in a reasonable time. The problem is that the requeing to the interrupt thread adds back in the same type of transition boundary you were trying to take out by getting rid of NETISR in the first place. -- Terry
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3F83BF33.46696D6C>