Date: Wed, 13 Mar 2002 16:19:48 -0800 From: Terry Lambert <tlambert2@mindspring.com> To: Greg Johnson <gjohnson@research.canon.com.au> Cc: mark tinguely <tinguely@web.cs.ndsu.nodak.edu>, freebsd-hackers@FreeBSD.ORG Subject: Re: DMA memory allocation/deallocation Message-ID: <3C8FECA4.CF3C293F@mindspring.com> References: <20020313204049.13A80540D0@brixi.research.canon.com.au>
next in thread | previous in thread | raw e-mail | index | archive | help
Greg Johnson wrote: > Thanks for the reply Mark, I tracked my problem down. In > 'bus_dmamem_alloc', 'contigmalloc' is used if the size is > greater than PAGE_SIZE. However, in 'bus_dmamem_free' > the same PAGE_SIZE check is performed, but nothing is > done for the case where the size is greater than PAGE_SIZE. > > So I then called contigfree explicitly in my code, and the > memory is released. > > Another problem though, I am allocating a large DMAable buffer > ~300MB. I get to allocate this and free this twice on our system > (512MB memory). However, when I try a third time to load my device > driver and allocate the memory, it fails. Subsequent attempts > also fail. Yes. This will happen because the memory becomes fragmented. If you allocated a huge block early, then the allocation can always be satisfied. If you deallocate it, then the system is free to use it up. So it can break up the previous space to use for smaller spaces. A subsequent attempt at allocation can therefore not be satisfied using the same memory, and the allocation may succeed, but it will do so at another location, after the allocations. At the very least, I can allocate the memory, make several small allocations, free the memory, and then make several small allocations, which then leaves me with needing to do the alocation after the set of small allocations after the first allcoation. If you are goung to be loading and unloading a driver during developement, and you need a huge gob of memory like this, then the correct approach is to do the allocation at or as soon after boot time as possible, and then reference the allcoated area from your load/unload/load/unload'ed driver. > It seems that something is allocating in the middle of my > nice big free buffer. Sure. You freed it. Anything that wants memory can alloc that area. Most likely, the allocation is proceeding at the start of the region as new allocations take place after the free. > Does 'contigalloc' do anything sensible like shuffling arround > of pages in physical memory to make room for the requested amount > of space? You mean "will it defrag physical memory" so that you can do another contiguous allocation, and "will it evict clean pages from physical memory to leave a large contiguous space" that can subsequently be used by an allocation. The answer is "no". What you are trying to do sould occur rarely, and the memory should be committed eary, so that this is never an issue in real life. If you want to fix this problem, you can write a physical memory defragmentation routine. It will probably stall most of the operations on your system, unless you seperate the reserve/commit phases of the malloc, so that you can incrementally reserve the memory as you defragthe map, so that it won't refrag behind you. Another approach for some systems is to seperate the memory, and do small allocations at the bottom and large ones at the top, such that when you need to do a large one, there is still space at the top for the allocation, without causing fragmentation when you fre and reallocate it several times. This approach, "expectation zoning for size", is in conflict with another, generally more useful approach, called "expectation zoning for persistance". In the second approach, you seperate memory into allocation pools by expectation of how lon an allocation will be held. By doing this, you get a relatively homogeneous lifetime for the memory in each zone. In practice, this will reduce overall fragmentation of the memory map up to 85% (or better, depending on your zone granularity). But it means that large allocations in any given zone are less likely to be successful. If you want to work on defragging physical memory, it's possbile to do, but will be difficult. The first thing that you will need to ensure is that the defragging code is not subject to its own operations. The second is that the space reclaiming code, which can only evict clean pages, or evict to swap dirty pages, doesn't touch any code or data in the paging path at all, or it won't work. In general, the ability to do this was expected to come into FreeBSD with the ELF switchover, but so far the only benefit is that there is an on average saving of 1/2 a page of disk at an increased cost of one page for the code/data duplication split for the combined code/data page. Actually, this was probably the wong way to do the trade, considering disk sizes are still growing faster than RAM, and RAM is limited to 32 bit for a while still). If you want to work on the code, to do this, I'm sure there are people who would be willing to help you out, after you take the lead on some code in that area. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3C8FECA4.CF3C293F>