Date: Wed, 22 Jan 2014 22:49:23 -0800 From: Navdeep Parhar <np@FreeBSD.org> To: Garrett Wollman <wollman@freebsd.org> Cc: net@freebsd.org Subject: Re: Use of contiguous physical memory in cxgbe driver Message-ID: <20140123064923.GA6501@ox> In-Reply-To: <21216.36928.132606.318491@hergotha.csail.mit.edu> References: <21216.22944.314697.179039@hergotha.csail.mit.edu> <52E06104.70600@FreeBSD.org> <21216.36928.132606.318491@hergotha.csail.mit.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, Jan 22, 2014 at 10:45:04PM -0500, Garrett Wollman wrote: > <<On Wed, 22 Jan 2014 16:23:32 -0800, Navdeep Parhar <np@FreeBSD.org> said: > > > On 01/22/14 15:52, Garrett Wollman wrote: > >> At this point everyone is well aware that requiring contiguous > >> physical page when the hardware can do scatter-gather is a very bad > >> idea. > > > I wouldn't put it this way. Using buffers with size > PAGE_SIZE has its > > advantages. > > These advantages do not come close to balancing out the disadvantage > of "your server eventually falls off the network due to physmem > fragmentation, better hope you can reset it remotely because driving > in to work at 3 AM sucks." This seems to imply that the only alternate to a successful allocation of the preferred size (which may be > PAGE_SIZE) is to not allocate any rx buffer at all and thus fall off the network eventually. This is a false choice. The driver can always fall back to allocating PAGE_SIZE sized buffers iff larger allocations fail. I'm considering implementing something along these lines in cxgbe(4), with exponential backoffs to avoid repeated allocation attempts from jumbo zones that are depleted. Regards, Navdeep > If any free pages are available at all, the allocation of a 4k jumbo > mbuf will succeed. A 9k jumbo mbuf requires three physically > contiguous pages, and it's very, very easy for physical memory to get > fragmented to the point where that is impossible. > > -GAWollman
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20140123064923.GA6501>