From owner-freebsd-net@FreeBSD.ORG Fri Sep 12 07:51:03 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3F92E6BE for ; Fri, 12 Sep 2014 07:51:03 +0000 (UTC) Received: from mx11.netapp.com (mx11.netapp.com [216.240.18.76]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "mx11.netapp.com", Issuer "VeriSign Class 3 International Server CA - G3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 1A1B18EE for ; Fri, 12 Sep 2014 07:51:02 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.04,511,1406617200"; d="asc'?scan'208";a="147240790" Received: from hioexcmbx06-prd.hq.netapp.com ([10.122.105.39]) by mx11-out.netapp.com with ESMTP; 12 Sep 2014 00:51:02 -0700 Received: from HIOEXCMBX07-PRD.hq.netapp.com (10.122.105.40) by hioexcmbx06-prd.hq.netapp.com (10.122.105.39) with Microsoft SMTP Server (TLS) id 15.0.913.22; Fri, 12 Sep 2014 00:50:52 -0700 Received: from HIOEXCMBX07-PRD.hq.netapp.com ([::1]) by hioexcmbx07-prd.hq.netapp.com ([fe80::79b6:a9a2:f48b:f065%21]) with mapi id 15.00.0913.011; Fri, 12 Sep 2014 00:50:52 -0700 From: "Eggert, Lars" To: Luigi Rizzo Subject: Re: netmap wishlist Thread-Topic: netmap wishlist Thread-Index: AQHPzk61Hrt2PZpnokKRDfZWlEftWpv9j9GAgAAFf4A= Date: Fri, 12 Sep 2014 07:50:52 +0000 Message-ID: <6D7C4630-98FA-45C9-A8DE-357065E47F0C@netapp.com> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-mailer: Apple Mail (2.1878.6) x-originating-ip: [10.122.56.79] Content-Type: multipart/signed; boundary="Apple-Mail=_7B1F0C82-0090-4A4D-984A-12DB8F989C5E"; protocol="application/pgp-signature"; micalg=pgp-sha1 MIME-Version: 1.0 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Sep 2014 07:51:03 -0000 --Apple-Mail=_7B1F0C82-0090-4A4D-984A-12DB8F989C5E Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Hi, On 2014-9-12, at 9:31, Luigi Rizzo wrote: > there is something already available/in progress for some of the = above, > but here are my thoughts on the various subjects: >=20 > - netmap is designed to work with large frames, by setting the buffer > size to something suitable (using a sysctl). ... > The downside is some waste on buffers (they are fixed size so = having > to allocate say 16K for a 64 byte frame is a bit annoying). that's OK for what I'm doing. > - checksums offloading can be added trivially in the *_txsync(), > once again on a per-nic basis. > Problem is, is we start adding per-packet features (say, checksums, > scatter-gather I/O, segmentation) in the inner loop of *_txsync() > we are going to lose some performance for high rate applications. What about making these things compile-time options? I totally see that = if you want to use netmap for fast switching, you wouldn't want these. = But if you use netmap for operating on IP and transport protocol = packets, they become really essential. (Esp. at 40G - which reminds me = that I forgot to add netmap support for the ixl driver to the = wishlist...) > - the VALE switch has support for segmentation and checksum avoidance. > Clients can register as virtio-net capable: in this case the port = will > accept/deliver large segments across that port, and do segmentation = and > checksum as required for ports that are not virtio-net enabled > (e.g. physical NICs attached to the same VALE switch). > This was developed earlier this year by Vincenzo Maffione. I may look into this. I'm unclear if adding a VALE layer into the system = just to get this feature would be wort it in terms of performance. > We could probably leverage this code to work also on top of NICs > connected through netmap, e.g. programming the NIC to use its own > native offloading, but i am skeptical about the usefulness and > concerned about the potential performance loss in *_txsync(). I totally see that, but maybe a compile-time option would work. There = are several distinct use cases for netmap at the moment, and it's = unlikely that the same system would need to support several of them, so = compile-time specialization may be sufficient here. > - Stefano Garzarella has some code to do software GSO (this is for = FreeBSD, > linux already has something similar), which will be presented at > EuroBSDCon later this month in Sofia. This should address the > segmentation issue on the host stack. Nice, I will take a look. > - on the receive side, both FreeBSD and Linux have an efficient > RLO software fallback in case the NIC does not support it > natively, i think we do not need this at the NIC/switch level. OK, I need to look into this. Oh, and my list was prioritized - I think the checksum offload would be = the real winner when dealing munging IP and transport packets. Thanks, Lars --Apple-Mail=_7B1F0C82-0090-4A4D-984A-12DB8F989C5E Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="signature.asc" Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- iQCVAwUBVBKl5NZcnpRveo1xAQLDdgQAsLU+tmwjVgEeFBChqSq+74yyMZSc7sOU ggV/qLBaazcY0514o5P3KoHCKU9qLIytZ6+u5Me4TckUgJ3YADxEH9TzYW7drs7t qQpUtaJcqFs7tV+BoBU7gNBFxa9+7mki5LBrDSiTMmRiKzmowijBTqDEN7lKvSX6 UaJk0tJYgVE= =bSa6 -----END PGP SIGNATURE----- --Apple-Mail=_7B1F0C82-0090-4A4D-984A-12DB8F989C5E--