From owner-freebsd-net@freebsd.org Fri Sep 16 15:05:36 2016 Return-Path: <owner-freebsd-net@freebsd.org> Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 41B69BDCCAD for <freebsd-net@mailman.ysv.freebsd.org>; Fri, 16 Sep 2016 15:05:36 +0000 (UTC) (envelope-from rizzo.unipi@gmail.com) Received: from mail-wm0-x229.google.com (mail-wm0-x229.google.com [IPv6:2a00:1450:400c:c09::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D26869CB for <freebsd-net@freebsd.org>; Fri, 16 Sep 2016 15:05:35 +0000 (UTC) (envelope-from rizzo.unipi@gmail.com) Received: by mail-wm0-x229.google.com with SMTP id 1so47020375wmz.1 for <freebsd-net@freebsd.org>; Fri, 16 Sep 2016 08:05:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=dMbbtSVVT+nvcb4UrDGLKmbkyS1A31YHbP45UN0j0HA=; b=xzYMXLi+bnynLCr/Er1K0Ca0l4YmYZz5VphscMFvmZlT/BR/upc58MerZ9RSeZwkr8 iamvv4qNsww6iCi6UN6BiiXmdBbbS9Heja308ftqcBWWgTSW6g51rnYo7F4iPSRLblp0 w82J70LWV4B5LbcyKXn1sbBjKTTm1TETo1wyqJm4gf1mr6wsTg94lKwRlRNu2o4nRQa5 ORIFNA8NeFEgvaEEr5dkaeAMbS0jshKt1wjW+CuWDTPdl2fkgdm/z9fH5d9PMAoIQZUK qbBajXZvJ0MpIjZ8mPxZk80yb+8i5dCX31GSstMliM3Lu2+LCZDb+OF7SMzDXz3U7mBB vb3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=dMbbtSVVT+nvcb4UrDGLKmbkyS1A31YHbP45UN0j0HA=; b=fWyaIPSHXKk+ejDx8tkGyACWFCb+CL5GhpN+WFf94z8jjQU1hLfn6ZR7s5b6LC64YT iScSfsRYtGHE+Q66WB//GKN0h+4wpKOf+28I63wcQxV/hnRlJP62BZKQ89lVF3UTA3P0 lLzQIpM5zte6pAdKjIrnA6xB7Zkbn1AlvpLi3ot2jBhNuak8SP3nA/8J3oRJdBYbDOn8 nRC8tcwdT5+KDjx1yiknPYPibYqMlAN+jP3UjS/OgKLQORq/gug8NY2ZjcosN04Vnzno 12LM8utQhDH+M4DDdfHHqmN4ErZNicr7okg+Y6rlpAMp3oPjEzmP627K77TCJopm2v2x UHCw== X-Gm-Message-State: AE9vXwODWAFSPlmOpkZ4sZ8pZ9j+QCNC5XZlJucK48zhEmmpEKoHqrQUu59jzj5DG4av1VjvQv9M4qxKnpJkLw== X-Received: by 10.28.109.86 with SMTP id i83mr8019344wmc.104.1474038334349; Fri, 16 Sep 2016 08:05:34 -0700 (PDT) MIME-Version: 1.0 Sender: rizzo.unipi@gmail.com Received: by 10.28.113.206 with HTTP; Fri, 16 Sep 2016 08:05:33 -0700 (PDT) In-Reply-To: <20160916145752.GC2840@zxy.spb.ru> References: <20160916145752.GC2840@zxy.spb.ru> From: Luigi Rizzo <rizzo@iet.unipi.it> Date: Fri, 16 Sep 2016 17:05:33 +0200 X-Google-Sender-Auth: Uib1wBfsPcRBZG0zGVgnttrvQCY Message-ID: <CA+hQ2+j2BfbA7A+M-JQt-bEvxnkGRYLZ9Dsz7Ojd1EzhjCBkkA@mail.gmail.com> Subject: Re: NETMAP application architecture question To: Slawa Olhovchenkov <slw@zxy.spb.ru> Cc: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org>, Giuseppe Lettieri <g.lettieri@iet.unipi.it>, Vincenzo Maffione <v.maffione@gmail.com>, Luigi Rizzo <rizzo@iet.unipi.it> Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org> List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-net>, <mailto:freebsd-net-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net/> List-Post: <mailto:freebsd-net@freebsd.org> List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help> List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-net>, <mailto:freebsd-net-request@freebsd.org?subject=subscribe> X-List-Received-Date: Fri, 16 Sep 2016 15:05:36 -0000 Hi, I don't think there is any hard limitation on the total number of pipes, very likely the number (64) is just some arbitrary limit that we set, and can be increased through a sysctl or worst case a compile time constant. Please have a look at the source code if you can find where you are hitting the limit, and which parameters are involved. In case you don't find it, get back to us in a week (we are busy on a deadline at the moment). Note that you should look at the netmap code on github, the one in the freebsd tree is not up to date. Cheers Luigi On Fri, Sep 16, 2016 at 4:57 PM, Slawa Olhovchenkov <slw@zxy.spb.ru> wrote: > I am wrote NETMAP application and meet some limitation. > I am use server 16-core server. > Workers thread run mostly independend, with minimal interworking. > I am use balancer threads for distributing packet flow between worker > threads. > I am using two NETMAP pipes between balancer and workers. > As result, for case of 4 balancer and 12 workers I am need 4x12x2 = 96 > pipes. NETMAP have limitation of 64 pipes. > > May be exist more optimal way? > > May be I can emulate tilera way, with bufers credits? > (In this case balancer give RX buffers, confirm to kernel and pass it > to worker. kernel reduce credit, used for reciving other buffer from > pool. After complete worker pass buffer to balancer and balancer > return buffer to kernel, kernel rise credits and put free buffer to > pool). > In this case I am not need to many pipes and not limited performance > by too long processing of one packet. > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" -- -----------------------------------------+------------------------------- Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione http://www.iet.unipi.it/~luigi/ . Universita` di Pisa TEL +39-050-2217533 . via Diotisalvi 2 Mobile +39-338-6809875 . 56122 PISA (Italy) -----------------------------------------+-------------------------------