Skip site navigation (1)Skip section navigation (2)
Date:      Fri, 28 Feb 2020 22:39:58 +0300
From:      Slawa Olhovchenkov <slw@zxy.spb.ru>
To:        Vincenzo Maffione <vmaffione@freebsd.org>
Cc:        FreeBSD Net <freebsd-net@freebsd.org>
Subject:   Re: Intel NETMAP performance and packet type
Message-ID:  <20200228193958.GR8012@zxy.spb.ru>
In-Reply-To: <CA%2B_eA9iZE9=fFFhOTo7mo3DN5mWErcP1dpDrf4_tLgAoqVPdEw@mail.gmail.com>
References:  <20200203204447.GD8028@zxy.spb.ru> <CA%2B_eA9it5iB7ERiDx0b7zN54atPZz64wW75nWeZrqTD58SWyLQ@mail.gmail.com> <20200225150924.GM8012@zxy.spb.ru> <CA%2B_eA9hqsUj7bXnq0%2BprqsxTr7_f9WE-5%2BJwUA==Xy0PvXAG=Q@mail.gmail.com> <20200227201650.GO8012@zxy.spb.ru> <CA%2B_eA9iZE9=fFFhOTo7mo3DN5mWErcP1dpDrf4_tLgAoqVPdEw@mail.gmail.com>

next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, Feb 28, 2020 at 06:31:36PM +0100, Vincenzo Maffione wrote:

> Il giorno gio 27 feb 2020 alle ore 21:17 Slawa Olhovchenkov <slw@zxy.spb.ru>
> ha scritto:
> 
> > On Thu, Feb 27, 2020 at 06:51:54PM +0100, Vincenzo Maffione wrote:
> >
> > > Hi,
> > >   So, the issue is not the payload.
> > > If you look at the avg_batch statistics reported by pkt-gen, you'll see
> > > that in the ACK-flood experiment you have 4.92, whereas in the SYN-flood
> > > case you have 17.5. The batch is the number of packets (well, actually
> > > netmap descriptors, but in this case it's the same) that you receive (or
> > > transmit) for each poll() invocation.
> > > So in the first case you end up doing much more poll() calls, hence the
> > > higher per-packet overhead and the lower packet-rate.
> > >
> > > Why is the poll() called more frequently? That depends on packet timing
> > and
> > > interrupt rate. There must be something different on your packet
> > generator
> > > that produces this effect (e.g. different burstiness, or maybe the packet
> > > generator is not able to saturate the 10G link)?
> >
> > No, I am capture netstat output -- raw packet rate is the same.
> > Also, I am change card to chelsio T5 and don't see issuse.
> >
> > This is payload issuse, at driver level.
> >
> 
> That's not possible, since netmap does not even look into the payload.

Netmap don't look, NIC can look.
I mean driver don't coretly/filly switch off all acceleration and ACK
flood caused badly handled event in Intel driver. Don't see for chelsio.

> Can you please report the per-queue interrupt rate in both cases (ACK-flood
> and SYN-flood)?
> You can use something like `vmstat -i -w1 | grep ix` to monitor the
> interrupt rate.
> Or probably you can also use `sysctl -a dev.ix | grep interrupt_rate`
> 
> 
> > > In any case, I would suggest measuring the RX interrupt rate, and check
> > > that it's higher in the ACK-flood case. Then you can try to lower the
> > > interrupt rate by tuning the interrupt moderation features of the Intel
> > NIC
> > > (e,g. limit hw.ix.max_interrupt_rate and disable hw.ix.enable_aim or
> > > similar).
> > > By playing with the interrupt moderation you should be able to increase
> > the
> > > avg_batch, and then increase throghput.
> >
> > Already limited.
> >
> 
> Limited to which value? Have you tried to decrease max_interrupt_rate even
> more?

16000, don't try different values for this case.



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20200228193958.GR8012>