From owner-freebsd-net@freebsd.org  Fri Sep 16 14:57:57 2016
Return-Path: <owner-freebsd-net@freebsd.org>
Delivered-To: freebsd-net@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1E790BDCA76
 for <freebsd-net@mailman.ysv.freebsd.org>;
 Fri, 16 Sep 2016 14:57:57 +0000 (UTC) (envelope-from slw@zxy.spb.ru)
Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id D9F993AF
 for <freebsd-net@freebsd.org>; Fri, 16 Sep 2016 14:57:56 +0000 (UTC)
 (envelope-from slw@zxy.spb.ru)
Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD))
 (envelope-from <slw@zxy.spb.ru>) id 1bkuaO-000LWo-Ki
 for freebsd-net@freebsd.org; Fri, 16 Sep 2016 17:57:52 +0300
Date: Fri, 16 Sep 2016 17:57:52 +0300
From: Slawa Olhovchenkov <slw@zxy.spb.ru>
To: freebsd-net@freebsd.org
Subject: NETMAP application architecture question
Message-ID: <20160916145752.GC2840@zxy.spb.ru>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.5.24 (2015-08-30)
X-SA-Exim-Connect-IP: <locally generated>
X-SA-Exim-Mail-From: slw@zxy.spb.ru
X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.23
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net/>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 16 Sep 2016 14:57:57 -0000

I am wrote NETMAP application and meet some limitation.
I am use server 16-core server.
Workers thread run mostly independend, with minimal interworking.
I am use balancer threads for distributing packet flow between worker
threads.
I am using two NETMAP pipes between balancer and workers.
As result, for case of 4 balancer and 12 workers I am need 4x12x2 = 96
pipes. NETMAP have limitation of 64 pipes.

May be exist more optimal way?

May be I can emulate tilera way, with bufers credits?
(In this case balancer give RX buffers, confirm to kernel and pass it
to worker. kernel reduce credit, used for reciving other buffer from
pool. After complete worker pass buffer to balancer and balancer
return buffer to kernel, kernel rise credits and put free buffer to
pool).
In this case I am not need to many pipes and not limited performance
by too long processing of one packet.