Date: Fri, 19 Feb 2010 22:48:32 +0400 From: rihad <rihad@mail.ru> To: freebsd-net@freebsd.org Subject: Slow speeds experienced with Dummynet Message-ID: <4B7EDD00.5080308@mail.ru>
next in thread | raw e-mail | index | archive | help
Hi, all, Recalling my old posting "dummynet dropping too many packets" dated October 4, 2009, the problem isn't over just yet. This time, there are no interface i/o drops (just a reminder: we have 2 bce(4) GigE cards connected to a Cisco router, one for input, and one for output. The box itself does some traffic accounting and enforces speed limits w/ ipfw/dummynet. There are normally around 5-6k users online). Traffic shaping is accomplished by this ipfw rule: pipe tablearg ip from any to table(0) out where table(0) contains those 5-6k IP addresses. The pipes themselves are GRED (or taildrop, it doesn't matter): ipfw pipe 512 config bw 512kbit/s mask dst-ip 0xffffffff gred 0.002/900/1000/0.1 queue 1000 Taking this template the speeds range from 512 to tens of mbps. With the setup as above very many users complain about very slow downloads, slow browsing. systat -ifstat, refreshed every 5 seconds, does reveal large differences between subsequent displays: if around 800-850 mbps is what's to be expected, it doesn't stay within those limits, also jumping to as low as 620+, and to somewhere in the 700's, Now imagine this: once I turn dummynet off (by short-circuiting a "allow ip from any to any" before the "pipe tablearg" rule) all user complaints stop, with traffic load staying stably at around 930-950 mbps. Does this have anything to do with "dummynet bursts"? How can I beat that? If I keep the pipe queue size at 2000 slots, the net.inet.ip.dummynet.io_pkt_drops sysctl stops increasing, once I start tweaking the value to as low as 100 slots, the value starts raising constantly at about 300-500 pps. I had hoped that smaller queue sizes would battle the negative effects of dummynet burstiness, it did so, I guess, but not in a very decisive manner. FreeBSD 7.1-RELEASE-p10 kern.hz=4000 kern.ipc.nmbclusters=111111 net.inet.ip.fastforwarding=1 net.inet.ip.dummynet.io_fast=1 net.isr.direct=0 net.inet.ip.intr_queue_maxlen=5000 net.inet.ip.dummynet.hash_size=512 net.inet.ip.dummynet.max_chain_len=16 net.inet.ip.intr_queue_drops: 0 systat -ip shows zero output drops at times of trouble. netstat -s's "output packets dropped due to no bufs, etc." is also fine. netstat -m shows nothing suspicious. p.s: Two "bloody expensive" Intel 10 GigE's are on their way to us to replace the Broadcom cards, meanwhile what should I try doing? Thanks for reading.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4B7EDD00.5080308>