Date: Fri, 16 Sep 2016 19:20:58 +0300 From: Slawa Olhovchenkov <slw@zxy.spb.ru> To: Luigi Rizzo <rizzo@iet.unipi.it> Cc: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org>, Giuseppe Lettieri <g.lettieri@iet.unipi.it>, Vincenzo Maffione <v.maffione@gmail.com> Subject: Re: NETMAP application architecture question Message-ID: <20160916162058.GE2840@zxy.spb.ru> In-Reply-To: <CA%2BhQ2%2Bj2BfbA7A%2BM-JQt-bEvxnkGRYLZ9Dsz7Ojd1EzhjCBkkA@mail.gmail.com> References: <20160916145752.GC2840@zxy.spb.ru> <CA%2BhQ2%2Bj2BfbA7A%2BM-JQt-bEvxnkGRYLZ9Dsz7Ojd1EzhjCBkkA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, Sep 16, 2016 at 05:05:33PM +0200, Luigi Rizzo wrote: > Hi, > I don't think there is any hard limitation on the total number > of pipes, very likely the number (64) is just some arbitrary > limit that we set, and can be increased through a sysctl or > worst case a compile time constant. > > Please have a look at the source code if you can find where > you are hitting the limit, and which parameters are involved. > In case you don't find it, get back to us in a week (we are > busy on a deadline at the moment). > > Note that you should look at the netmap code on github, the one in > the freebsd tree is not up to date. Next about tilera-like. Can I allocate some buffers for custom-made pools? I.e. balancer thread do some like: src = &src_ring->slot[rxr->cur]; free_buf_idx = next_free(private_pool); to_worker.idx = src->buf_idx; to_worker.len = src->len; src->buf_idx = tmp; src->flags = NS_BUF_CHANGED; rxr->head = rxr->cur = nm_ring_next(rxr, rxr->cur); to_worker pass to worker w/o netmap. balancer put to_worker in to private_pool after worker done. I am need reserved about x64 of one ring size per balancer. i.e. about 131072 buffers per balancer and about 512K total buffers. How I can do this? > On Fri, Sep 16, 2016 at 4:57 PM, Slawa Olhovchenkov <slw@zxy.spb.ru> wrote: > > I am wrote NETMAP application and meet some limitation. > > I am use server 16-core server. > > Workers thread run mostly independend, with minimal interworking. > > I am use balancer threads for distributing packet flow between worker > > threads. > > I am using two NETMAP pipes between balancer and workers. > > As result, for case of 4 balancer and 12 workers I am need 4x12x2 = 96 > > pipes. NETMAP have limitation of 64 pipes. > > > > May be exist more optimal way? > > > > May be I can emulate tilera way, with bufers credits? > > (In this case balancer give RX buffers, confirm to kernel and pass it > > to worker. kernel reduce credit, used for reciving other buffer from > > pool. After complete worker pass buffer to balancer and balancer > > return buffer to kernel, kernel rise credits and put free buffer to > > pool). > > In this case I am not need to many pipes and not limited performance > > by too long processing of one packet. > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > > > -- > -----------------------------------------+------------------------------- > Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione > http://www.iet.unipi.it/~luigi/ . Universita` di Pisa > TEL +39-050-2217533 . via Diotisalvi 2 > Mobile +39-338-6809875 . 56122 PISA (Italy) > -----------------------------------------+-------------------------------
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20160916162058.GE2840>