Date: Tue, 21 Jun 2005 21:11:57 +0200 From: Pieter de Boer <pieter@thedarkside.nl> To: Luigi Rizzo <rizzo@icir.org> Cc: freebsd-net@freebsd.org Subject: Re: Issues with a Large Fat pipe Network simulation Message-ID: <42B8667D.9020504@thedarkside.nl> In-Reply-To: <20050621102954.A66904@xorpc.icir.org> References: <42B722EF.2090203@thedarkside.nl> <20050620135044.B35720@xorpc.icir.org> <42B731CD.1040104@thedarkside.nl> <20050621075247.D63359@xorpc.icir.org> <42B84AC8.7050802@thedarkside.nl> <20050621102954.A66904@xorpc.icir.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Luigi Rizzo wrote: > oh yes one thing... you are using 'via foo0' in your rule, > which means the packet is intercepted both in the input and > output path, which causes further contention on the queues. Well, when using 'ip from client to server recv em0', packets get matched twice. When I set some latency on that pipe, the packets incur double the latency I set. With 'via em0' this isn't the case. I tried it out, but didn't seem to make much difference. > also you can set the queue size in kbytes, and can probably go as high > as 1000kb. maybe this helps too. Doesn't seem to do much either. > I am pretty sure there is some issue there, also related to some > timing issues and tcp window opening mode (slow start vs. linear) I went to see if there were any sysctl's I could tune a bit. I found these: net.inet.ip.intr_queue_maxlen: 50 net.inet.ip.intr_queue_drops: 382136 I don't like drops. So I set intr_queue_maxlen to 400, and -poof-, the speed went up to around 700mbit/s. Still not as fast as it was with 64KB send/recv spaces, but it's a huge improvement nonetheless. I guess we probably should tune a bit more until we're confident that the middle-box behaves correctly, before adding things like latency and packet-loss :) Thanks for the advice! If you know other settings to tune on the dummynetting host, I'd very much like to hear them. I'm pondering about polling (which means we can't do SMP on the dummynet system, but it's only pushing packets, so that shouldn't matter too much). At least we have somewhat more info to work with now :) -- Pieter
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?42B8667D.9020504>