Date: Thu, 20 Mar 2014 18:34:04 -0300 From: Christopher Forgeron <csforgeron@gmail.com> To: Garrett Wollman <wollman@hergotha.csail.mit.edu> Cc: freebsd-net@freebsd.org Subject: Re: 9.2 ixgbe tx queue hang Message-ID: <CAB2_NwCjQ6qtx0SbEERVGs2Y_5pae-g=UbFEwkzmWYGoTuoP%2BQ@mail.gmail.com> In-Reply-To: <201403202113.s2KLD7GB085085@hergotha.csail.mit.edu> References: <CAB2_NwDG=gB1WCJ7JKTHpkJCrvPuAhipkn%2BvPyT%2BxXzOBrTGkg@mail.gmail.com> <1159309884.25490921.1395282576806.JavaMail.root@uoguelph.ca> <CAB2_NwAOmPtZjB03pdDiTK2OvQgqk-tYf83Jq4Ukt9jnZA8CNA@mail.gmail.com> <201403202113.s2KLD7GB085085@hergotha.csail.mit.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
I have found this: http://lists.freebsd.org/pipermail/freebsd-net/2013-October/036955.html I think what you're saying is that; - a MTU of 9000 doesn't need to equal a 9k mbuf / jumbo cluster - modern NIC drivers can gather 9000 bytes of data from various memory locations - The fact that I'm seeing 9k jumbo clusters is showing me that my driver is trying to allocate 9k of contiguous space, and it's failing. Please correct me if I'm off here, I'd love to understand more. On Thu, Mar 20, 2014 at 6:13 PM, Garrett Wollman < wollman@hergotha.csail.mit.edu> wrote: > In article > <CAB2_NwAOmPtZjB03pdDiTK2OvQgqk-tYf83Jq4Ukt9jnZA8CNA@mail.gmail.com>, > csforgeron@gmail.com writes: > > >50/27433/0 requests for jumbo clusters denied (4k/9k/16k) > > This is going to screw you. You need to make sure that no NIC driver > ever allocates 9k jumbo pages -- unless you are using one of those > mythical drivers that can't do scatter/gather DMA on receive, which > you don't appear to be. > > These failures occur when the driver is trying to replenish its > receive queue, but is unable to allocate three *physically* contiguous > pages of RAM to construct the 9k jumbo cluster (of which the remaining > 3k is simply wasted). This happens on any moderately active server, > once physical memory gets checkerboarded with active single pages, > particularly with ZFS where those pages are wired in kernel memory and > so can't be evicted. > > -GAWollman >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAB2_NwCjQ6qtx0SbEERVGs2Y_5pae-g=UbFEwkzmWYGoTuoP%2BQ>