Date: Mon, 7 Oct 2002 11:52:15 -0700 (PDT) From: Bill Baumann <bbaumann@isilon.com> To: freebsd-net@FreeBSD.ORG Subject: Re: Help with net.inet.ip.intr_queue_maxlen Message-ID: <Pine.BSF.4.21.0210071105390.51262-100000@isilon.com> In-Reply-To: <3D9F8002.70500@expertcity.com>
next in thread | previous in thread | raw e-mail | index | archive | help
My thought is that the net.inet.ip.intr_queue_maxlen should be set to the maximum receive queue length of your NIC. Depending upon traffic (bursty short packets being the worst), other kernel operations being slow, and NIC interrupt coalesce or polling times, your NIC's rx buffer can fill significantly before the NIC's receive queue is be processed. Thats why we have large queues at this level! Unfortuantely, the default for net.inet.ip.intr_queue_maxlen is not nearly large enough to to accomidate but small percentage of the NICs rx queue. The default is 50, but should be more like 500. I think the bge driver's rx queue can handle 512 1500MTU plus 256 9000MTU frames. So the NIC can receive 768 before dropping frames, but the IP stack can only handle the first 50. As far as I know none of the drivers know to do anything about this problem. Due to the above, I can image scenarios where you would experience significant packet loss with a low CPU utilization. You may also want to consider decreasing interrupt coalescing parameters in the NIC or polling times. This will lower the latencies, but increase the CPU overhead attributed to the NIC. Good luck! On Sat, 5 Oct 2002, Steve Francis wrote: > Can someone help me with net.inet.ip.intr_queue_maxlen tuning? > > Firstly, its the "size of the IP input queue", per the source. > > So does that mean after the NIC has received the packet, the interupt > from the NIC has been processed and the packet retrieved from the NIC, > then the packet is placed in this queue, before the IP stack looks at it? > i.e. its unrelated to interupt coalescing or polling, or NIC > performance, as they have already occurred in order to put the packet > into the queue. Yes? > > I am getting incrementing net.inet.ip.intr_queue_drops at around 8,000 > pps (increasing drops at rate of 10 or so per second.) > Yet, if my statement above about what the queue is, is correct, then it > just means that the system was busy doing stuff before it had a chance > to process the incoming packets, so there was no room for new ones to > enter the queue. But as the system was only 50% busy, then if I increase > the input queue, I should be able to avoid these drops, correct? At > least until the system gets a lot busier. > > Is there a sane upper recommended limit to the queue length? > > Or am I way off base here? > Thanks > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-net" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-net" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.BSF.4.21.0210071105390.51262-100000>