Date: Sun, 14 Oct 2001 01:50:28 -0700 From: Terry Lambert <tlambert2@mindspring.com> To: Mike Silbersack <silby@silby.com> Cc: freebsd-current@freebsd.org Subject: Re: Some interrupt coalescing tests Message-ID: <3BC951D4.E9638595@mindspring.com> References: <20011014014546.L36700-100000@achilles.silby.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Mike Silbersack wrote: > Hm, true, I guess the improvement is respectable. My thought is mostly > that I'm not sure how much it's extending the performance range of a > system; testing with more varied packet loads as suggested by Alfred would > help tell us the answer to this. I didn't respond to Alfred's post, and I probably should have; he had some very good comments, including varying the load. My main interest has been in increasing throughput as much as possible; as such, my packet load has been geared towards moving the most data possible. The tests we did were with just connections per second, 1k HTTP transfers, and 10k HTTP transfers. Unfortunately, I can't give you seperate numbers without the LRP, since we didn't bother after the connection rate went from ~7000/second to 23500/second with LRP, it wasn't worth it. > The extra polling of the bus in cases where there are no additional > packets to grab is what I was wondering about. I guess in comparison to > the quantity of packet data going by, it's not a real issue. It could be, if you were doing something that was network triggered, relatively low cost, but CPU intensive; on the whole, though, there's very little that isn't going to be network related, these days, and what there is, will end up not taking the overhead, unless you are also doing networking. Maybe it should be a tunable? But these days, everything is pretty much I/O bound, not CPU bound. The one thing I _would_ add -- though I'm waiting for it to be a problem before doing it -- is to limit the total number of packets processed per interrupt by keeping a running count. You would have to be _AMAZINGLY_ loaded to hit this, though; since it would mean absolutely continuous DMAs. I think it is self-limiting, should that happen, since once you are out of mbufs, you're out. The correct thing to do is probably to let it run out, but keep a seperate transmit reserve, so that you can process requests to completion. I don't know if anyone has tested what happens to apache in a denial of service attack consisting of a huge number of partial "GET" requests that are incomplete, and so leave state hanging around in the HTTP server... [ ... polling vs. interrupt load ... ] > Straight polling isn't necessarily the solution I was thinking of, but > rather some form of interrupt disabling at high rates. For example, if > the driver were to keep track of how many interrupts/second it was taking, > perhaps it could up the number of receive buffers from 64 to something > higher, then disable the card's interrupt and set a callback to run in a > short bit of time at which point interrupts would be reenabled and the > interrupt handler would be run. Ideally, this could reduce the number of > interrupts greatly, increasing efficiency under load. Paired with this > could be receive polling during transmit, something which does not seem to > be done at current, if I'm reading correctly. > > I'm not sure of the feasibility of the above, unfortunately - it would > seem highly dependant on how short of a timeout we can realistically get > along with how many mbufs we can spare for receive buffers. Yes. Floyd and Druschel recommend using high and low watermarks on the amount of data pending processing in user space. The most common approach is to use a fair share scheduling algorithm, which reserves a certain amount of CPU for user space processing, but this is somewhat wasteful, if there is no work, since it denies quantum to the interrupt processing, potentially wrongly. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-current" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3BC951D4.E9638595>