Date: Tue, 30 Dec 2014 19:40:34 +0100 From: Luigi Rizzo <rizzo@iet.unipi.it> To: Carlos Ferreira <carlosmf.pt@gmail.com> Cc: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org> Subject: Re: Regarding Netmap internal memory allocation. Message-ID: <CA%2BhQ2%2Bij-7wZrp%2BoLpVnTPFe-FNJV_ro8JxYQdPBTfKP2BMyPA@mail.gmail.com> In-Reply-To: <CAJpYY6U=x_PAf0pKoWZTQwCfb0_kmra4Qb4TiUKgXKeTphXidw@mail.gmail.com> References: <CAJpYY6XmOnYX4bm=YyXr5hoiFLXhoEFXDa44Z_Fn1fHNj8G5_w@mail.gmail.com> <CAJpYY6UH4bU9xYBsAgZm6_h1no9TDPJ9J2TNp4V19rYGrRGuuw@mail.gmail.com> <CA%2BhQ2%2BiD=2g65ScRBzVVYGRu=jmxksCtMNmObTTnnXfG=Yuu3g@mail.gmail.com> <CAJpYY6UGuzVY%2B=_m2RhOfhR40NHxv6uM%2BfWeJpOGkj9DDXWKiw@mail.gmail.com> <CAJpYY6Xy1AZi%2Bb7Dm%2Bn-TnvJJzWEjzQC35%2BntbLm2f4DST=yjQ@mail.gmail.com> <CAJpYY6WGO%2BW8%2BpPfNk1G9WUAO_6DVZf5iqbw-nFP7wnhfxBHQw@mail.gmail.com> <CA%2BhQ2%2Bj5JkNn6H9cEdWtsfdv7B5XMto2TQgpYk%2Bvky7v0QQheA@mail.gmail.com> <CAJpYY6U=x_PAf0pKoWZTQwCfb0_kmra4Qb4TiUKgXKeTphXidw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Dec 30, 2014 at 6:38 PM, Carlos Ferreira <carlosmf.pt@gmail.com> wrote:
> Ok, I'm having some trouble in tuning the amount of memory for netmap.
>
> I have been following the man page from FreeBSD in other to understand the
> values at /sys/modules/netmap/parameters for linux but I'm having some
> trouble in understanding what each value actually means.
> For the following values:
>
> dev.netmap.ring_num: 200 -> Is this the number of rings in the Ring Buffer
> Pool?
yes. For interfaces with a single queue you need 4 rings (rx+tx for the nic and
another rx+tx for the host port)
> dev.netmap.ring_size: 36864 -> Is this value, the number of slots per ring?
this is the size in bytes of each ring. The number of slots is set by the
hardware (low end devices as in the openwrt devices will probably use
256 or 512 slots, so 10-12k should suffice. But this is not worth changing.
Instead, you should reduce the number of buffers, though 8MB is only 4000
buffers and it is a bit on the low side for 5 ports.
However, as far as I know most openwrt devices only have one physical NIC,
and a switch implementing various vlans.
cheers
luigi
>
> I'm trying to keep the amount of memory used by netmap as low as 4MB - 8MB
> since I'm going to use only up to 4 NICs and one TAP.
>
> Thanks for the help!
>
>
> On 30 December 2014 at 16:16, Luigi Rizzo <rizzo@iet.unipi.it> wrote:
>>
>> you can #undefine WITH_VALE.
>> But it is only 20K of code (and 150K of data structures, which you
>> can further reduce by lowering NM_BRIDGS).
>> The saving is probably not worth the effort.
>>
>> cheers
>> luigi
>>
>> On Tue, Dec 30, 2014 at 5:08 PM, Carlos Ferreira <carlosmf.pt@gmail.com>
>> wrote:
>> > By the way, another question.
>> > Is there a way to not compile the code regarding the VALE switch? I'm
>> > only
>> > interested in using netmap with Tap Devices and NICs, so I was hoping to
>> > save some memory.
>> >
>> > On 30 December 2014 at 15:47, Carlos Ferreira <carlosmf.pt@gmail.com>
>> > wrote:
>> >>
>> >> You mean netmap_mem2.c ? It was there where I found the
>> >> NETMAP_BUF_MAX_NUM
>> >> define.
>> >>
>> >>
>> >>
>> >> On 30 December 2014 at 15:43, Carlos Ferreira <carlosmf.pt@gmail.com>
>> >> wrote:
>> >>>
>> >>> Ok thanks. I was hoping not having to recompile the module, but it's
>> >>> ok.
>> >>> Thank you for the info!
>> >>>
>> >>>
>> >>> On 30 December 2014 at 15:38, Luigi Rizzo <rizzo@iet.unipi.it> wrote:
>> >>>>
>> >>>> you can reduce the amount of ram (buffers, mostly) by
>> >>>> tweaking the values in netmap_mem2.c ::
>> >>>> struct netmap_obj_params netmap_params[NETMAP_POOLS_NR] = {
>> >>>> ...
>> >>>> }
>> >>>>
>> >>>> or you can simply modify the constant
>> >>>>
>> >>>> netmap_mem2.h:#define NETMAP_BUF_MAX_NUM 20*4096*2
>> >>>>
>> >>>> to something smaller that suits an openwrt box
>> >>>> (in which i am very interested, as I'd like to deploy one of these
>> >>>> soon)
>> >>>>
>> >>>> cheers
>> >>>> luigi
>> >>>>
>> >>>>
>> >>>> On Tue, Dec 30, 2014 at 4:12 PM, Carlos Ferreira
>> >>>> <carlosmf.pt@gmail.com>
>> >>>> wrote:
>> >>>> > Update:
>> >>>> >
>> >>>> > I noticed that the netmap module was still crashing, after
>> >>>> > changing
>> >>>> > the
>> >>>> > OpenWRT VM ram to 256MB. I now raised to 1GB and it no longer
>> >>>> > crashed.
>> >>>> > The
>> >>>> > netmap module is now consuming about 350MB of Ram, which for my
>> >>>> > objectives
>> >>>> > is just too much...
>> >>>> >
>> >>>> > On 30 December 2014 at 14:06, Carlos Ferreira
>> >>>> > <carlosmf.pt@gmail.com>
>> >>>> > wrote:
>> >>>> >
>> >>>> >> To Luigi and to whom may be able to help
>> >>>> >>
>> >>>> >> Hello all.
>> >>>> >>
>> >>>> >> Is it possible to reduce the size of the memory buffer allocated
>> >>>> >> by
>> >>>> >> the
>> >>>> >> netmap module?
>> >>>> >> I'm asking this because I was implementing some testing code,
>> >>>> >> using
>> >>>> >> NICs
>> >>>> >> and a Tap device in an OpenWRT VM with 64MB of RAM.
>> >>>> >> Because of the small RAM amount, the nm_open crashed when the
>> >>>> >> program
>> >>>> >> tried to netmap the tap device, after I previously netmapped one
>> >>>> >> NIC
>> >>>> >> successfully.
>> >>>> >> After the crash, I bumped the VM RAM to 256MB and the test program
>> >>>> >> ran
>> >>>> >> well, but not without me noticing that the VM RAM consumption was
>> >>>> >> increased about 90 MB by netmap.
>> >>>> >>
>> >>>> >> Resuming, I want to know if there is a way to reduce the memory
>> >>>> >> buffer
>> >>>> >> allocation, without recompiling the netmap kernel module.
>> >>>> >>
>> >>>> >> Thank you for the attention.
>> >>>> >>
>> >>>> >> --
>> >>>> >>
>> >>>> >> Carlos Miguel Ferreira
>> >>>> >> Researcher at Telecommunications Institute
>> >>>> >> Aveiro - Portugal
>> >>>> >> Work E-mail - cmf@av.it.pt
>> >>>> >> Skype & GTalk -> carlosmf.pt@gmail.com
>> >>>> >> LinkedIn -> http://www.linkedin.com/in/carlosmferreira
>> >>>> >>
>> >>>> >
>> >>>> >
>> >>>> >
>> >>>> > --
>> >>>> >
>> >>>> > Carlos Miguel Ferreira
>> >>>> > Researcher at Telecommunications Institute
>> >>>> > Aveiro - Portugal
>> >>>> > Work E-mail - cmf@av.it.pt
>> >>>> > Skype & GTalk -> carlosmf.pt@gmail.com
>> >>>> > LinkedIn -> http://www.linkedin.com/in/carlosmferreira
>> >>>> > _______________________________________________
>> >>>> > freebsd-net@freebsd.org mailing list
>> >>>> > http://lists.freebsd.org/mailman/listinfo/freebsd-net
>> >>>> > To unsubscribe, send any mail to
>> >>>> > "freebsd-net-unsubscribe@freebsd.org"
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>>
>> >>>>
>> >>>> -----------------------------------------+-------------------------------
>> >>>> Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing.
>> >>>> dell'Informazione
>> >>>> http://www.iet.unipi.it/~luigi/ . Universita` di Pisa
>> >>>> TEL +39-050-2211611 . via Diotisalvi 2
>> >>>> Mobile +39-338-6809875 . 56122 PISA (Italy)
>> >>>>
>> >>>>
>> >>>> -----------------------------------------+-------------------------------
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>>
>> >>> Carlos Miguel Ferreira
>> >>> Researcher at Telecommunications Institute
>> >>> Aveiro - Portugal
>> >>> Work E-mail - cmf@av.it.pt
>> >>> Skype & GTalk -> carlosmf.pt@gmail.com
>> >>> LinkedIn -> http://www.linkedin.com/in/carlosmferreira
>> >>
>> >>
>> >>
>> >>
>> >> --
>> >>
>> >> Carlos Miguel Ferreira
>> >> Researcher at Telecommunications Institute
>> >> Aveiro - Portugal
>> >> Work E-mail - cmf@av.it.pt
>> >> Skype & GTalk -> carlosmf.pt@gmail.com
>> >> LinkedIn -> http://www.linkedin.com/in/carlosmferreira
>> >
>> >
>> >
>> >
>> > --
>> >
>> > Carlos Miguel Ferreira
>> > Researcher at Telecommunications Institute
>> > Aveiro - Portugal
>> > Work E-mail - cmf@av.it.pt
>> > Skype & GTalk -> carlosmf.pt@gmail.com
>> > LinkedIn -> http://www.linkedin.com/in/carlosmferreira
>>
>>
>>
>> --
>> -----------------------------------------+-------------------------------
>> Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione
>> http://www.iet.unipi.it/~luigi/ . Universita` di Pisa
>> TEL +39-050-2211611 . via Diotisalvi 2
>> Mobile +39-338-6809875 . 56122 PISA (Italy)
>> -----------------------------------------+-------------------------------
>
>
>
>
> --
>
> Carlos Miguel Ferreira
> Researcher at Telecommunications Institute
> Aveiro - Portugal
> Work E-mail - cmf@av.it.pt
> Skype & GTalk -> carlosmf.pt@gmail.com
> LinkedIn -> http://www.linkedin.com/in/carlosmferreira
--
-----------------------------------------+-------------------------------
Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione
http://www.iet.unipi.it/~luigi/ . Universita` di Pisa
TEL +39-050-2211611 . via Diotisalvi 2
Mobile +39-338-6809875 . 56122 PISA (Italy)
-----------------------------------------+-------------------------------
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CA%2BhQ2%2Bij-7wZrp%2BoLpVnTPFe-FNJV_ro8JxYQdPBTfKP2BMyPA>