Date: Thu, 8 Dec 2011 19:29:32 +0100 From: Luigi Rizzo <rizzo@iet.unipi.it> To: John Baldwin <jhb@FreeBSD.org> Cc: svn-src-head@freebsd.org, Luigi Rizzo <luigi@freebsd.org>, src-committers@freebsd.org, svn-src-all@freebsd.org, Marius Strobl <marius@alchemy.franken.de> Subject: Re: svn commit: r228281 - in head/sys/dev: e1000 re Message-ID: <20111208182932.GA82488@onelab2.iet.unipi.it> In-Reply-To: <4EE0F89E.1070107@FreeBSD.org> References: <201112051533.pB5FXEuh043063@svn.freebsd.org> <20111205183854.GC54475@alchemy.franken.de> <20111205193150.GB49118@onelab2.iet.unipi.it> <4EE0F89E.1070107@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, Dec 08, 2011 at 12:49:18PM -0500, John Baldwin wrote: > On 12/5/11 2:31 PM, Luigi Rizzo wrote: > >On Mon, Dec 05, 2011 at 07:38:54PM +0100, Marius Strobl wrote: > >>On Mon, Dec 05, 2011 at 03:33:14PM +0000, Luigi Rizzo wrote: > >... > >>>+#ifdef DEV_NETMAP > >>>+ if (slot) { > >>>+ int si = i + na->tx_rings[txr->me].nkr_hwofs; > >>>+ void *addr; > >>>+ > >>>+ if (si>= na->num_tx_desc) > >>>+ si -= na->num_tx_desc; > >>>+ addr = NMB(slot + si); > >>>+ txr->tx_base[i].buffer_addr = > >>>+ htole64(vtophys(addr)); > >>>+ /* reload the map for netmap mode */ > >>>+ netmap_load_map(txr->txtag, > >>>+ txbuf->map, addr, na->buff_size); > >>>+ } > >>>+#endif /* DEV_NETMAP */ > >> > >>Can these vtophys(9) usages be fixed to use bus_dma(9) instead so netmap > >>works with bounce buffers, IOMMUs etc? > > > >maybe. Can you suggest how to change it ? > > > >Consider that (not here but in other places) vtophys() is called > >in a time-critical loop so performance matters a lot. As long as i > >can compute the physical address in advance and cache it in my own > >array, i suppose that should be fine (in which case the calls to > >vtophys(addr) would become NMPB(slot + si) where the NMPB() macro > >would hide translations and checks. > > For your use case, you probably don't want to be coping with bounce > buffers at all. That is, if you are preallocating long-lived buffers > that keep getting reused while netmap is active that are allocated at > startup and free'd at teardown, you probably want to allocate buffers > that won't require bounce buffers. That means you have to let the > drivers allocate the buffers (or give you a suitable bus_dma tag since > different devices have different addressing requirements, etc.). You > could then use bus_dmamem_alloc() to allocate your buffers. certainly i don't want to use netmap with bounce buffers. I am not sure about IOMMU (I basically don't need it but maybe using a compatible API is always nice). Right now i am allocating a huge chunk of memory with contigmalloc. Ryan Stone suggested that a plain malloc may work as well (as long as i make sure that each buffer is within a single page). Eventually I may want to play with cache alignment (also suggested by Ryan) so allocate smaller chunks of contigmalloc'ed memory (say each buffer is 2K - 64 bytes, then a contiguous block of 64K fits exactly 33 buffers). cheers luigi may also work as long as i make sure th > > -- > John Baldwin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20111208182932.GA82488>