From owner-freebsd-net@freebsd.org Sat Feb 18 09:36:12 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BA60ACE2A55 for ; Sat, 18 Feb 2017 09:36:12 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 964011D49 for ; Sat, 18 Feb 2017 09:36:12 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 9282DCE2A54; Sat, 18 Feb 2017 09:36:12 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 921C7CE2A53 for ; Sat, 18 Feb 2017 09:36:12 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: from mail-ot0-x22d.google.com (mail-ot0-x22d.google.com [IPv6:2607:f8b0:4003:c0f::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 548F51D48 for ; Sat, 18 Feb 2017 09:36:12 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: by mail-ot0-x22d.google.com with SMTP id 32so45167051oth.3 for ; Sat, 18 Feb 2017 01:36:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=xX/rSGZ/RNAHENhnXOFlSeruuJnIv9Uv+o+vYWUhpUI=; b=RkA49myvAOrBTBce2E5AQgnUlC5uiPOCUf6Qp+3uuCHbSdZVoCZeLtb5RUdrEXqqo5 NVSuQ9BejS17tamqwQsgeoMoNNLl+W7db/tKK1GLgmd5EwSTtxhOdACeNHiei7gMMLL8 mAxpo91/TBWnb6kF9i3cTnxAEHljPu4WhKRg0nT0fOZl2UJjnnhkeNmkvpNndR3s5c9d 3079Sr6pBIZKvqndY7vqAdB27hCeE72WfY8voadLn60egtGIW+MYKbgA071a0ANASWWt 26d8dlwwJ5uNg6Be1CMlWannq83IZ7Udi0rI+20uTi5k/vaM+YHl7eafT/zvFMcZkTd+ LgMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=xX/rSGZ/RNAHENhnXOFlSeruuJnIv9Uv+o+vYWUhpUI=; b=ZYTC8Pc87CmxZEPzzd6iE/dcqDcir2RAm96Fo0NlfvbH/FLdrBtNoxDqcxGru0jkT2 nCJKdjaNzNzzjy+U9Znq/QQeJ+kg40NSNC5epIWpc7/uFnrddZS9ZARMK3ZDTrGJq63P LoFNsPVpo0dDUQ2Fm+bY6D3JxkWZ9HR+ZJmofdaULtrhfCF8Nzcmdn12LUQs4ikYjxPc YBTf2LKPwwBg3NIsciDbQAz3FFOYDeIpgU5JNdijsbH7/oP6Xsv8QZscGgKvXejbv0we e91byZYLP1Vn5ptKyjUDgf4cwa5sLlwcfc5tJrHlDg4D1PUsQ4nH4m54E+b7R58RES7f sG1g== X-Gm-Message-State: AMke39k/F6/jkNrhX5XPA5bFfDOkbdc9HLvwI276X91oEsHS0JlaJZMzaZ7VgfMEBd9vy5FxFjHOf+1i3XzQpQ== X-Received: by 10.157.40.46 with SMTP id m43mr7659435otb.213.1487410571546; Sat, 18 Feb 2017 01:36:11 -0800 (PST) MIME-Version: 1.0 Received: by 10.157.7.201 with HTTP; Sat, 18 Feb 2017 01:36:11 -0800 (PST) In-Reply-To: <20170216205819.GS6035@zxy.spb.ru> References: <20170214123621.GA15630@zxy.spb.ru> <20170216203813.GR6035@zxy.spb.ru> <20170216205819.GS6035@zxy.spb.ru> From: Vincenzo Maffione Date: Sat, 18 Feb 2017 10:36:11 +0100 Message-ID: Subject: Re: netmap_mem_get_info To: Slawa Olhovchenkov Cc: amir alipour , "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Feb 2017 09:36:12 -0000 For each netmap memory area (in this case you use only the global one): - if_num must be >= of the total number of NIOCREGIF on the netmap memory area. The default is 100 and it should be ok. - Then you have to count how many TX and RX rings you use in total in all the interfaces you NIOCREGIF'd in the memory area. So using your terminology, in the worst case you have (K + K+1 + 1) rings (NIC Tx+ NIC rx + host tx + host rx) from the outside interface and (L+L+1+1) rings for the inside interface. You also have 2 rings for each couple of pipes you are using. So ring_num must be >= than the rings count. - Then you need to count the total number of slots in all those rings. Each ring may have differenet number of descriptors, so you need to check it out that for each interface (e.g. "ethtool -g ethX" on linux, probably some sysctl on freebsd). Pipes by default have the same number of slots of their parent adapter. If x is the total number of slots, then it must be buf_num >= x. - Finally, you have to look for the ring(s) that has the maximum number of slots. If that number is y, then it must be ring_size >= sizeof(struct netmap_slot)*y + sizeof(struct netmap_ring). Cheers, Vincenzo 2017-02-16 21:58 GMT+01:00 Slawa Olhovchenkov : > On Thu, Feb 16, 2017 at 09:48:14PM +0100, Vincenzo Maffione wrote: > > > Not sure about what you mean. Until memory areas are in use the real > values > > (*_num, *_size) are not changed. > > At NIOCREGIF time you can say what allocator you are interested in by > > writing a non-zero id inside req.nr_arg2. > > My application have N balancer thread and M worker thread. > Application serve outside interface w/ K rings (and ring size N) and > inside interface w/ L rings (and ring size M). > > All balancer threads share rings from interfaces in next maner: > > first thread open descriptors to 1st ring of inside and outside interface > next thread open descriptors to 2nd ring of inside and outside interface > > and etc in round-roubin. > > every balancer thread open 2 pipe to every worker thread. > > how many if_num, ring_num, buf_num I am need to configure? > > > 2017-02-16 21:38 GMT+01:00 Slawa Olhovchenkov : > > > > > On Thu, Feb 16, 2017 at 09:14:19PM +0100, Vincenzo Maffione wrote: > > > > > > > Hi, > > > > You're right, we'll try to add more details. > > > > > > > > In any case, buf_size, ring_size and if_size are the sizes in bytes > of > > > each > > > > buffer, ring and netmap_if (control data structure), respectively. > > > > So the maximum amount of slots for each ring is ring_size/16, as 16 > is > > > the > > > > size in bytes of struct netmap_slot. > > > > > > > > On the other side, buf_num, ring_num and if_num are the total number > of > > > > biffers, rings and netmap_if objects in each netmap memory area (aka > > > > "allocator"). > > > > By default there is a single memory area used by all the hardware > NICs > > > and > > > > a separate memory area for each VALE port. > > > > This is already configurable, however. > > > > > > This values also depends on open netmap descriptors, right? > > > > > > > 2017-02-14 13:36 GMT+01:00 Slawa Olhovchenkov : > > > > > > > > > On Tue, Feb 14, 2017 at 12:26:55PM +0100, Vincenzo Maffione wrote: > > > > > > > > > > > Hi, > > > > > > Have you tried to play with netmap sysctl parameters like: > > > > > > > > > > > > dev.netmap.buf_num > > > > > > dev.netmap.ring_num > > > > > > dev.netmap.if_num > > > > > > > > > > > > those are listed in the sysctl section of the netmap man page. > > > > > > > > > > man page hide details about calcul rules of this parameters. > > > > > > > > > > > > > > > > > > > > > -- > > > > Vincenzo Maffione > > > > > > > > > > > -- > > Vincenzo Maffione > -- Vincenzo Maffione