Date: Fri, 27 Dec 2002 19:19:35 +0100 From: Thomas Moestl <tmoestl@gmx.net> To: Harti Brandt <brandt@fokus.gmd.de> Cc: sparc@freebsd.org Subject: Re: busdma on sparc Message-ID: <20021227181934.GA290@crow.dom2ip.de> In-Reply-To: <20021227121312.I56345-100000@beagle.fokus.gmd.de> References: <20021227121312.I56345-100000@beagle.fokus.gmd.de>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, 2002/12/27 at 12:53:23 +0100, Harti Brandt wrote: > I have a problem with porting my driver for the Marconi HE ATM card to > sparc: > > The driver may (dependend on its cofiguration) need to give a lot of > mbufs and mbuf clusters to the card for its different receive queues. In > the biggest configuration I tried on i386 I have around 2000 clusters and > 4000 mbufs. Hmmm, so the application is to handle a lot of small packets arriving (or being sent) at high frequencies, right? What do you need the clusters for in this case? Such a high number of maps will also use up quite some kernel memory, both for the maps themselves and for resource management. I would suggest to hack up the mbuf allocator to make it possible to allocate contiguous arrays of mbufs (e.g. one page at a time, which would amount to 16 to 32 mbufs). This would cut the number of 4000 mbuf maps to 125 on sparc64. Of course, this would also require some more management on the driver's part, since mbufs that were handed up cannot be replaced immediately (which should however not be a problem from the latency point of view). There might also be a fragmentation problem, which could be partly worked around with another mh_flag to tell the allocator to put it back into a contiguous pool if possible. > In iommu.c each dmamap eats up 32k of virtual address space (16k > because of the page size and 16k for alignment). The page size is just 8kB. The alignment depends on your driver, the IOMMU code just enforces alignment on a page boundary (i.e. no additional alignment, since allocations are always on page boundaries). This means that just 16kB are reserved for small DMA maps. The additional 8kB are needed to handle cases where a buffer spans a page boundary. This may not happen with mbufs, but can of course occur in other situations. > This means, that each mbuf of lesser than 200 bytes eats 32k of > virtual DMA address space. This makes it impossible to allocate > enough buffers to the card for high performance applications. I was > under the impression, that mbufs are already aligned to every > possible alignment requirements, so wouldn't it be possible to > reduce the 32k to 16k on request from the driver (perhaps via one of > the BUS_DMA_BUS[1-4] flags? I do not think that using one of the flags would be a good idea, since it requires drivers to do MD things to an MI API. In any event the maximum number of pages with the current IOMMU TSB size is 4096, so this would not help you for the largest configuration. Increasing the TSB size to 64kB or 128kB might be another option for you. I am working on a revision of the IOMMU code right now, I'll look into a better way to get around this. - Thomas -- Thomas Moestl <tmoestl@gmx.net> http://www.tu-bs.de/~y0015675/ <tmm@FreeBSD.org> http://people.FreeBSD.org/~tmm/ PGP fingerprint: 1C97 A604 2BD0 E492 51D0 9C0F 1FE6 4F1D 419C 776C To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-sparc" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20021227181934.GA290>