Date: Mon, 27 Sep 2004 15:07:27 EDT From: TM4525@aol.com To: stheg_olloydson@yahoo.com Cc: freebsd-questions@freebsd.org Subject: Re: Device polling performance Message-ID: <1c8.1f3de3fe.2e89beef@aol.com>
next in thread | raw e-mail | index | archive | help
In a message dated 9/25/04 4:24:07 PM Eastern Daylight Time, stheg_olloydson@yahoo.com writes: >The EVIDENCE is to the contrary, since it seems that a 2.4Ghz system >will be saturated when bridging ~250Kpps with device-polling enabled, >based on polling stats and userland benchmarking, even though the >system claims to be 100% idle. Interestingly, its about the same with >interrupt enabled. > >The POINT is that since there is no way to measure the performance, >you've got a bunch of guys who think they've figured something out >touting device-polling without having a clue what the performance >advantages (or consequences) are, so it might as well be black magic, >or snake oil, since you are as blind as a bat in your assessments. Hello, Please post your "polling stats and userland benchmarking" results. I would be very interested seeing them as I was thinking of moving to NICs that would benefit from polling. However, because you have "EVIDENCE ... to the contrary", I may hold off. On the other hand, you do go on to say "there is no way to measure the performance" and "you are as blind as a bat in your assessments", so also please post your test methodology. I need to make my decision on reliable, repeatable facts. Also, when you post, would you please wrap your lines to a shorter length? Not everyone on the list uses AOL Reader, like you. ------------------------------------------------------------------------------ --------------------------- The "evidence" is a bit circumstantial in the absence of working tools, but here are some "observations". There's also an assumption that the knobs associated with polling work as expected. Test machine is a 2.4Ghz celeron box with dual Intel NICs (em driver) on a 32bit, 33Mhz bus, running FreeBSD 4.9. Now I realize that a 133Mhz, 64bit bus is 8x faster and you certainly wouldnt use these NICs on a real network, but for the purpose of a control it doesnt matter, since both tests are on the same MB. Settings: HZ=1000 each_burst=512 max_burst=1000 user_frac=variable RXdescriptors (receive ring size) = 512 (Note that the burst never exceeded 100 at any time) I'm firing a controlled stream of 100K pps through the box (bridging). With only normal userland (idle) usage, the box happily goes about its way. Top shows 0-1.5% usage. I started a cpu intensive userland task (buildworld or something of the sort). The system started to lose packets with a user_frac setting of 78, which implies that the system requires about 22% of the cpu to successfully manage the task, assuming the knob works (it appears to). The same machine, with interrupts enabled, uses about 26%, according to "top". HOWEVER, setting hz back to 100, with interrupts enabled the usage went down under 25%. Given that, it can be argued that there is less than a 5% bonus for polling, which makes a lot more sense than what some of the kooks have been saying. Of course the point here wasn't to prove the difference, which Im still not sure of, but the evidence certainly is that "top" doesn't properly account for CPU usage in device_polling mode.. I'd expect a small bonus, but nothing earth-shattering, as the machine still has to do the same amount of work. Its not like the machine is really servicing an interrupt for every packet, since controllers have hold offs so they don't generate interrupts on top of each other, and multiple events are regularly handled with a single interrupt. Polling gives the appearance of a machine happily going about its business no matter how much traffic you throw at it, but what happens is that you lose packets when it becomes overmatched, which never happens on a system with interrupts enabled before it goes into livelock. While livelock isn't a good thing, if it only happens occasionally, at least you aren't losing packets. Additionally, with a HZ setting of 1000 you're also introducing quite a bit of latency: additional_latency = up to 1ms in receive ring + transmit time for burst-1 frames. Increasing HZ further would reduce the latency, but adds more overhead, which slims the advantages and defeats the purpose of polling in the first place. The bottom line is that there isn't so much difference as to think that polling is going to save the day, but if you don't care about latency or losing packets it can be useful in allocating cpu cycles to user space, if thats your priority. TM
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1c8.1f3de3fe.2e89beef>