Date: Mon, 22 Jan 2001 13:26:47 -0600 From: Robert Lipe <robertlipe@usa.net> To: Alfred Perlstein <bright@wintelcom.net> Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: contigmalloc, M_WAITOK, & leaks. Message-ID: <20010122132647.I10504@rjlhome.sco.com> In-Reply-To: <20010122105227.E7240@fw.wintelcom.net>; from bright@wintelcom.net on Mon, Jan 22, 2001 at 10:52:27AM -0800 References: <20010122110642.B10504@rjlhome.sco.com> <20010122100524.D7240@fw.wintelcom.net> <20010122124539.F10504@rjlhome.sco.com> <20010122105227.E7240@fw.wintelcom.net>
next in thread | previous in thread | raw e-mail | index | archive | help
> > In this case, the resource failure isn't temporary. Once it starts > > failing, it fails until the system is rebooted. Since this is on a > > dedicated thread, I could handle it if it really was temporary. Of > > course, putting me to sleep so that someone else (or even another of my > > own threads) could run around and free the resources would be ideal. As > > it stands, once I get the failure notice, I'm hosed. > > Yup, it's sort of the fault of the memory allocator, afaik except > for allocations that are > (PAGE_SIZE/2), you can't free back to > the kernel map, meaning that if you allocate several thousand < > (PAGE_SIZE/2) chunks you've pretty much broken contigmalloc. Aaaah. There's a hint. Yes, by the time we get into trouble, I've allocated (and freed) several thousand chunks that are < PAGE_SIZE/2. This is happening in periods of a few dozen. (I don't know the number, but I think it's 48 or 64.) The test program allocates a few dozen tx buffers, fills them up, then the tx completion handler releases them. There are never very many of these outstanding at any time. All the allocs (in this specific case) are always the same size, and there are no intervening calls to contigmalloc. This should stack the deck pretty well for fragmentation. So if I sandbagged the allocs to be *larger* the KMA would behave more consistently with what I'd expect becuase it would then reclaim? I didn't see that one coming. :-) Would rounding them up to PAGE_SIZE/2 suffice, or did you really mean "> PAGE_SIZE/2"? > Not hinting much, the only reason I can guess that you need contigmalloc > is because you're allocating > PAGE_SIZE chunks and you don't want to > tell your hardware to do scatter/gather IO and would rather point it > at some physically contig memory and be done with it. Is there an s/g memory interface in FreeBSD? This was my first choice, but since I couldn't find a set of functions to build a list of buffers that satisfied a set of constraints, I fell back to contigmalloc to get things off the ground. I'd be delighted to use an interface to get me an s/g list with a given set of constraints that pointed to a list of buffers with a given set of constraints. > This is possible (using contigmalloc) if you cache the contigmalloc'd > chunks, but not if you keep discarding them and allow the kernel memory > map to become more fragmented. Implementing a cache for these is a few lines of code and will improve the robustness of what I have. I'm clearly going to have to do this. > My question is, why exactly do you need contigmalloc, is this some > device you're trying to write a driver for? Can you explain it some? > That would help me suggest a workaround. Sure. I'm trying to write a driver for arbitrary devices. :-) This is kinky, I know. UDI (www.projectudi.org) allows a driver writer to specify a very rich set of constraints on DMA objects. I'm implementing the piece of the UDI environment that provides this to the drivers. So while I know that the test program and driver I'm using right now don't *really* need contigmalloc, we'll eventually need the ability to ensure that DMA buffers for 32-bit PCI cards are in the bottom 32 bits, (or perhaps even not the bottom if you have intervening bus bridges) that stupid hardware providing 24 bits of addressing but bolted to a PCI controller gets allocated in the bottom 16Mb, etc. If I was trying to support a very specific piece of hardware, I could divine workarounds, but it's something I'm probably going to have to determine programmatically. Thanx again, RJL To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20010122132647.I10504>