Date: Mon, 4 May 2015 15:29:13 +0000 (UTC) From: Barney Cordoba <barney_cordoba@yahoo.com> To: Luigi Rizzo <rizzo@iet.unipi.it> Cc: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org> Subject: Re: Fwd: netmap-ipfw on em0 em1 Message-ID: <1009610346.1107538.1430753353703.JavaMail.yahoo@mail.yahoo.com> In-Reply-To: <CA%2BhQ2%2Bj7wPca%2Bf_h67u8aHN1BGdVDLvUeRGSXahbNooY6qc9kA@mail.gmail.com> References: <CA%2BhQ2%2Bj7wPca%2Bf_h67u8aHN1BGdVDLvUeRGSXahbNooY6qc9kA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
It's not faster than "wedging" into the if_input()s. It simply can't be. Yo=
ur getting packets at interrupt time as soon as their processed and =C2=A0y=
ou there's no network stack involved, and your able to receive and transmit=
without a process switch. At worst it's the same, without the extra plumbi=
ng. It's not rocket science to "bypass the network stack".
The only advantage of bringing it into user space would be that it's easier=
to write threaded handlers for complex uses; but not as a firewall (which =
is the limit of the context of my comment). You can do anything in the kern=
el that you can do in user space. The reason a kernel module with if_input(=
) hooks is better is that you can use the standard kernel without all of th=
e netmap hacks. You can just pop it into any kernel and it works.
BC=20
On Sunday, May 3, 2015 2:13 PM, Luigi Rizzo <rizzo@iet.unipi.it> wrote=
:
=20
On Sun, May 3, 2015 at 6:17 PM, Barney Cordoba via freebsd-net <
freebsd-net@freebsd.org> wrote:
> Frankly I'm baffled by netmap. You can easily write a loadable kernel
> module that moves packets from 1 interface to another and hook in the
> firewall; why would you want to bring them up into user space? It's 1000s
> of lines of unnecessary code.
>
>
Because it is much faster.
The motivation for netmap-like
solutions (that includes Intel's DPDK, PF_RING/DNA
and several proprietary implementations) is speed:
they bypass the entire network stack, and a
good part of the device drivers, so you can access
packets=20
10+ times faster.
So things are actually the other way around:
the 1000's of unnecessary
lines of code
(not really thousands, though)
are
those that you'd pay going through the standard
network stack
when you
don't need any of its services.
Going to userspace is just a side effect -- turns out to
be easier to develop and run your packet processing code
in userspace, but there are netmap clients (e.g. the
VALE software switch) which run entirely in the kernel.
cheers
luigi
>
>
>=C2=A0 =C2=A0 =C2=A0 On Sunday, May 3, 2015 3:10 AM, Raimundo Santos <rait=
ech@gmail.com>
> wrote:
>
>
>=C2=A0 Clarifying things for the sake of documentation:
>
> To use the host stack, append a ^ character after the name of the interfa=
ce
> you want to use. (Info from netmap(4) shipped with FreeBSD 10.1 RELEASE.)
>
> Examples:
>
> "kipfw em0" does nothing useful.
> "kipfw netmap:em0" disconnects the NIC from the usual data path, i.e.,
> there are no host communications.
> "kipfw netmap:em0 netmap:em0^" or "kipfw netmap:em0+" places the
> netmap-ipfw rules between the NIC and the host stack entry point associat=
ed
> (the IP addresses configured on it with ifconfig, ARP and RARP, etc...)
> with the same NIC.
>
> On 10 November 2014 at 18:29, Evandro Nunes <evandronunes12@gmail.com>
> wrote:
>
> > dear professor luigi,
> > i have some numbers, I am filtering 773Kpps with kipfw using 60% of CPU
> and
> > system using the rest, this system is a 8core at 2.4Ghz, but only one
> core
> > is in use
> > in this next round of tests, my NIC is now an avoton with igb(4) driver=
,
> > currently with 4 queues per NIC (total 8 queues for kipfw bridge)
> > i have read in your papers we should expect something similar to 1.48Mp=
ps
> > how can I benefit from the other CPUs which are completely idle? I trie=
d
> > CPU Affinity (cpuset) kipfw but system CPU usage follows userland kipfw
> so
> > I could not set one CPU to userland while other for system
> >
>
> All the papers talk about *generating* lots of packets, not *processing*
> lots of packets. What this netmap example does is processing. If someone
> really wants to use the host stack, the expected performance WILL BE wors=
e
> - what's the point of using a host stack bypassing tool/framework if
> someone will end up using the host stack?
>
> And by generating, usually the papers means: minimum sized UDP packets.
>
>
> >
> > can you please enlighten?
> >
>
> For everyone: read the manuals, read related and indicated materials
> (papers, web sites, etc), and, as a least resource, read the code. Within
> netmap's codes, it's more easy than it sounds.
> _______________________________________________
> freebsd-net@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
>
>
>
> _______________________________________________
> freebsd-net@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
>
--=20
-----------------------------------------+-------------------------------
Prof. Luigi RIZZO, rizzo@iet.unipi.it=C2=A0 . Dip. di Ing. dell'Informazio=
ne
http://www.iet.unipi.it/~luigi/ =C2=A0 =C2=A0 =C2=A0 . Universita` di Pisa
TEL=C2=A0 =C2=A0 =C2=A0 +39-050-2217533=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 . via Diotisalvi 2
Mobile=C2=A0 +39-338-6809875=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 . 56122 PISA (Italy)
-----------------------------------------+-------------------------------
_______________________________________________
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
From owner-freebsd-net@FreeBSD.ORG Mon May 4 15:52:44 2015
Return-Path: <owner-freebsd-net@FreeBSD.ORG>
Delivered-To: freebsd-net@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
(No client certificate requested)
by hub.freebsd.org (Postfix) with ESMTPS id 63FC4D33
for <freebsd-net@freebsd.org>; Mon, 4 May 2015 15:52:44 +0000 (UTC)
Received: from sola.nimnet.asn.au (paqi.nimnet.asn.au [115.70.110.159])
(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id B028B105C
for <freebsd-net@freebsd.org>; Mon, 4 May 2015 15:52:43 +0000 (UTC)
Received: from localhost (localhost [127.0.0.1])
by sola.nimnet.asn.au (8.14.2/8.14.2) with ESMTP id t44FqXmW079973;
Tue, 5 May 2015 01:52:33 +1000 (EST)
(envelope-from smithi@nimnet.asn.au)
Date: Tue, 5 May 2015 01:52:33 +1000 (EST)
From: Ian Smith <smithi@nimnet.asn.au>
To: Barney Cordoba <barney_cordoba@yahoo.com>
cc: Luigi Rizzo <rizzo@iet.unipi.it>,
"freebsd-net@freebsd.org" <freebsd-net@freebsd.org>
Subject: Re: Fwd: netmap-ipfw on em0 em1
In-Reply-To: <1009610346.1107538.1430753353703.JavaMail.yahoo@mail.yahoo.com>
Message-ID: <20150505014431.O26659@sola.nimnet.asn.au>
References: <CA+hQ2+j7wPca+f_h67u8aHN1BGdVDLvUeRGSXahbNooY6qc9kA@mail.gmail.com>
<1009610346.1107538.1430753353703.JavaMail.yahoo@mail.yahoo.com>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=ISO-8859-1
Content-Transfer-Encoding: 8BIT
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.20
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-net>,
<mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net/>;
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
<mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 04 May 2015 15:52:44 -0000
On Mon, 4 May 2015 15:29:13 +0000, Barney Cordoba via freebsd-net wrote:
> It's not faster than "wedging" into the if_input()s. It simply can't
> be. Your getting packets at interrupt time as soon as their processed
> and you there's no network stack involved, and your able to receive
> and transmit without a process switch. At worst it's the same,
> without the extra plumbing. It's not rocket science to "bypass the
> network stack".
> The only advantage of bringing it into user space would be that it's
> easier to write threaded handlers for complex uses; but not as a
> firewall (which is the limit of the context of my comment). You can
> do anything in the kernel that you can do in user space. The reason a
> kernel module with if_input() hooks is better is that you can use the
> standard kernel without all of the netmap hacks. You can just pop it
> into any kernel and it works.
Barney, do you have a working alternative implementation you can share
with us to help put this silly inferior netmap thingy out of business?
Thanks, Ian
[I'm sorry, pine doesn't quote messages from some yahoo users properly:]
On Sunday, May 3, 2015 2:13 PM, Luigi Rizzo <rizzo@iet.unipi.it> wrote:
On Sun, May 3, 2015 at 6:17 PM, Barney Cordoba via freebsd-net <
freebsd-net@freebsd.org> wrote:
> Frankly I'm baffled by netmap. You can easily write a loadable kernel
> module that moves packets from 1 interface to another and hook in the
> firewall; why would you want to bring them up into user space? It's 1000s
> of lines of unnecessary code.
>
>
Because it is much faster.
The motivation for netmap-like
solutions (that includes Intel's DPDK, PF_RING/DNA
and several proprietary implementations) is speed:
they bypass the entire network stack, and a
good part of the device drivers, so you can access
packets
10+ times faster.
So things are actually the other way around:
the 1000's of unnecessary
lines of code
(not really thousands, though)
are
those that you'd pay going through the standard
network stack
when you
don't need any of its services.
Going to userspace is just a side effect -- turns out to
be easier to develop and run your packet processing code
in userspace, but there are netmap clients (e.g. the
VALE software switch) which run entirely in the kernel.
cheers
luigi
>
>
> On Sunday, May 3, 2015 3:10 AM, Raimundo Santos <raitech@gmail.com>
> wrote:
>
>
> Clarifying things for the sake of documentation:
>
> To use the host stack, append a ^ character after the name of the interface
> you want to use. (Info from netmap(4) shipped with FreeBSD 10.1 RELEASE.)
>
> Examples:
>
> "kipfw em0" does nothing useful.
> "kipfw netmap:em0" disconnects the NIC from the usual data path, i.e.,
> there are no host communications.
> "kipfw netmap:em0 netmap:em0^" or "kipfw netmap:em0+" places the
> netmap-ipfw rules between the NIC and the host stack entry point associated
> (the IP addresses configured on it with ifconfig, ARP and RARP, etc...)
> with the same NIC.
>
> On 10 November 2014 at 18:29, Evandro Nunes <evandronunes12@gmail.com>
> wrote:
>
> > dear professor luigi,
> > i have some numbers, I am filtering 773Kpps with kipfw using 60% of CPU
> and
> > system using the rest, this system is a 8core at 2.4Ghz, but only one
> core
> > is in use
> > in this next round of tests, my NIC is now an avoton with igb(4) driver,
> > currently with 4 queues per NIC (total 8 queues for kipfw bridge)
> > i have read in your papers we should expect something similar to 1.48Mpps
> > how can I benefit from the other CPUs which are completely idle? I tried
> > CPU Affinity (cpuset) kipfw but system CPU usage follows userland kipfw
> so
> > I could not set one CPU to userland while other for system
> >
>
> All the papers talk about *generating* lots of packets, not *processing*
> lots of packets. What this netmap example does is processing. If someone
> really wants to use the host stack, the expected performance WILL BE worse
> - what's the point of using a host stack bypassing tool/framework if
> someone will end up using the host stack?
>
> And by generating, usually the papers means: minimum sized UDP packets.
>
>
> >
> > can you please enlighten?
> >
>
> For everyone: read the manuals, read related and indicated materials
> (papers, web sites, etc), and, as a least resource, read the code. Within
> netmap's codes, it's more easy than it sounds.
> _______________________________________________
> freebsd-net@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
>
>
>
> _______________________________________________
> freebsd-net@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
>
--
-----------------------------------------+-------------------------------
Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione
http://www.iet.unipi.it/~luigi/ . Universita` di Pisa
TEL +39-050-2217533 . via Diotisalvi 2
Mobile +39-338-6809875 . 56122 PISA (Italy)
-----------------------------------------+-------------------------------
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1009610346.1107538.1430753353703.JavaMail.yahoo>