From owner-freebsd-net@FreeBSD.ORG  Mon Mar 24 15:45:31 2014
Return-Path: <owner-freebsd-net@FreeBSD.ORG>
Delivered-To: freebsd-net@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTPS id E01AA793
 for <freebsd-net@freebsd.org>; Mon, 24 Mar 2014 15:45:31 +0000 (UTC)
Received: from smtp1.bushwire.net (f5.bushwire.net [199.48.133.46])
 by mx1.freebsd.org (Postfix) with SMTP id 91C263E5
 for <freebsd-net@freebsd.org>; Mon, 24 Mar 2014 15:45:29 +0000 (UTC)
Received: (qmail 30672 invoked by uid 1001); 24 Mar 2014 15:45:22 -0000
Delivered-To: qmda-intercept-freebsd-net@freebsd.org
DomainKey-Signature: a=rsa-sha1; q=dns; c=simple; s=s384; d=romeo.emu.st; 
 b=caS7RxAAMaZaHOsx3da03ZqFVrJdirapuMl0fv7yixU2dudeZYC13yPPk4Th240u;
Comments: DomainKeys? See http://en.wikipedia.org/wiki/DomainKeys
DomainKey-Trace-MD: h=10; b=30; l=C18R71D32M65F38T27S64M17C39C27;
Comments: QMDA 0.3
Received: (qmail 30665 invoked by uid 1001); 24 Mar 2014 15:45:22 -0000
Date: 24 Mar 2014 15:45:22 +0000
Message-ID: <20140324154522.30664.qmail@f5-external.bushwire.net>
From: "Mark Delany" <c2h@romeo.emu.st>
To: freebsd-net@freebsd.org
Subject: How to make netmap NS_FORWARD work with NR_REG_ONE_NIC?
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net/>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 24 Mar 2014 15:45:31 -0000

(Very recent fbsd10)

My application is taking advantage of NR_REG_ONE_NIC to register
separate handlers for each h/w ring. (Pro tip, you must re-open
/dev/netmap each time, a dup() fd doesn't work).

While good for performance, it unfortunately appears that NS_FORWARD
does not work in this mode - presumably because NR_REG_ONE_NIC doesn't
include the host ring.

Now it's possible I simply have a bug and NS_FORWARD should work with
NR_REG_ONE_NIC - I have demo code I can share if needed - but assuming
for a moment I don't have a bug...

What's the right way to implement host forwarding with NR_REG_ONE_NIC?

Do I have to register an independent handler with NR_REG_SW_NIC and
have the h/w ring handlers synchronize with it? If so, I also presume
the NR_REG_SW_NIC handler has to manage transfers in both directions,
yes?

I'm not keen on that approach if I can avoid it as it introduces
synchronization costs between handlers where previously I needed none.

I also looked at the kernel module and determined that nr_flags =
NR_REG_ONE_NIC | NR_REG_SW_NIC is invalid. Oh well. That would have
solved my problem nicely.


Mark.