From owner-svn-src-head@FreeBSD.ORG Thu Dec 8 18:13:24 2011 Return-Path: Delivered-To: svn-src-head@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4D79B106566B; Thu, 8 Dec 2011 18:13:24 +0000 (UTC) (envelope-from luigi@onelab2.iet.unipi.it) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id 0BCE78FC14; Thu, 8 Dec 2011 18:13:22 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id 7EF597300A; Thu, 8 Dec 2011 19:29:32 +0100 (CET) Date: Thu, 8 Dec 2011 19:29:32 +0100 From: Luigi Rizzo To: John Baldwin Message-ID: <20111208182932.GA82488@onelab2.iet.unipi.it> References: <201112051533.pB5FXEuh043063@svn.freebsd.org> <20111205183854.GC54475@alchemy.franken.de> <20111205193150.GB49118@onelab2.iet.unipi.it> <4EE0F89E.1070107@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4EE0F89E.1070107@FreeBSD.org> User-Agent: Mutt/1.4.2.3i Cc: svn-src-head@freebsd.org, Luigi Rizzo , src-committers@freebsd.org, svn-src-all@freebsd.org, Marius Strobl Subject: Re: svn commit: r228281 - in head/sys/dev: e1000 re X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Dec 2011 18:13:24 -0000 On Thu, Dec 08, 2011 at 12:49:18PM -0500, John Baldwin wrote: > On 12/5/11 2:31 PM, Luigi Rizzo wrote: > >On Mon, Dec 05, 2011 at 07:38:54PM +0100, Marius Strobl wrote: > >>On Mon, Dec 05, 2011 at 03:33:14PM +0000, Luigi Rizzo wrote: > >... > >>>+#ifdef DEV_NETMAP > >>>+ if (slot) { > >>>+ int si = i + na->tx_rings[txr->me].nkr_hwofs; > >>>+ void *addr; > >>>+ > >>>+ if (si>= na->num_tx_desc) > >>>+ si -= na->num_tx_desc; > >>>+ addr = NMB(slot + si); > >>>+ txr->tx_base[i].buffer_addr = > >>>+ htole64(vtophys(addr)); > >>>+ /* reload the map for netmap mode */ > >>>+ netmap_load_map(txr->txtag, > >>>+ txbuf->map, addr, na->buff_size); > >>>+ } > >>>+#endif /* DEV_NETMAP */ > >> > >>Can these vtophys(9) usages be fixed to use bus_dma(9) instead so netmap > >>works with bounce buffers, IOMMUs etc? > > > >maybe. Can you suggest how to change it ? > > > >Consider that (not here but in other places) vtophys() is called > >in a time-critical loop so performance matters a lot. As long as i > >can compute the physical address in advance and cache it in my own > >array, i suppose that should be fine (in which case the calls to > >vtophys(addr) would become NMPB(slot + si) where the NMPB() macro > >would hide translations and checks. > > For your use case, you probably don't want to be coping with bounce > buffers at all. That is, if you are preallocating long-lived buffers > that keep getting reused while netmap is active that are allocated at > startup and free'd at teardown, you probably want to allocate buffers > that won't require bounce buffers. That means you have to let the > drivers allocate the buffers (or give you a suitable bus_dma tag since > different devices have different addressing requirements, etc.). You > could then use bus_dmamem_alloc() to allocate your buffers. certainly i don't want to use netmap with bounce buffers. I am not sure about IOMMU (I basically don't need it but maybe using a compatible API is always nice). Right now i am allocating a huge chunk of memory with contigmalloc. Ryan Stone suggested that a plain malloc may work as well (as long as i make sure that each buffer is within a single page). Eventually I may want to play with cache alignment (also suggested by Ryan) so allocate smaller chunks of contigmalloc'ed memory (say each buffer is 2K - 64 bytes, then a contiguous block of 64K fits exactly 33 buffers). cheers luigi may also work as long as i make sure th > > -- > John Baldwin