From owner-freebsd-net@freebsd.org Fri Sep 16 15:14:48 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DE9ABBDCF37 for ; Fri, 16 Sep 2016 15:14:48 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 95BE4E01 for ; Fri, 16 Sep 2016 15:14:48 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1bkuqj-000Lvf-VA; Fri, 16 Sep 2016 18:14:46 +0300 Date: Fri, 16 Sep 2016 18:14:45 +0300 From: Slawa Olhovchenkov To: Luigi Rizzo Cc: "freebsd-net@freebsd.org" , Giuseppe Lettieri , Vincenzo Maffione Subject: Re: NETMAP application architecture question Message-ID: <20160916151445.GD2840@zxy.spb.ru> References: <20160916145752.GC2840@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Sep 2016 15:14:49 -0000 On Fri, Sep 16, 2016 at 05:05:33PM +0200, Luigi Rizzo wrote: > Hi, > I don't think there is any hard limitation on the total number > of pipes, very likely the number (64) is just some arbitrary > limit that we set, and can be increased through a sysctl or > worst case a compile time constant. I am don't see sysctl for this. Re-compile is not comfortable. In any way, for this setup I am need to many resources, as I am calc: buf_num=1114112 ring_num=964 if_num=248 This can introduce to high overhead. > Please have a look at the source code if you can find where > you are hitting the limit, and which parameters are involved. > In case you don't find it, get back to us in a week (we are > busy on a deadline at the moment). OK > Note that you should look at the netmap code on github, the one in > the freebsd tree is not up to date. Do you plan to update code in freebsd tree? > Cheers > Luigi > > On Fri, Sep 16, 2016 at 4:57 PM, Slawa Olhovchenkov wrote: > > I am wrote NETMAP application and meet some limitation. > > I am use server 16-core server. > > Workers thread run mostly independend, with minimal interworking. > > I am use balancer threads for distributing packet flow between worker > > threads. > > I am using two NETMAP pipes between balancer and workers. > > As result, for case of 4 balancer and 12 workers I am need 4x12x2 = 96 > > pipes. NETMAP have limitation of 64 pipes. > > > > May be exist more optimal way? > > > > May be I can emulate tilera way, with bufers credits? > > (In this case balancer give RX buffers, confirm to kernel and pass it > > to worker. kernel reduce credit, used for reciving other buffer from > > pool. After complete worker pass buffer to balancer and balancer > > return buffer to kernel, kernel rise credits and put free buffer to > > pool). > > In this case I am not need to many pipes and not limited performance > > by too long processing of one packet. > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > > > -- > -----------------------------------------+------------------------------- > Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione > http://www.iet.unipi.it/~luigi/ . Universita` di Pisa > TEL +39-050-2217533 . via Diotisalvi 2 > Mobile +39-338-6809875 . 56122 PISA (Italy) > -----------------------------------------+-------------------------------