From owner-freebsd-net@freebsd.org Wed Jan 11 18:45:50 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6F341CABA2B for ; Wed, 11 Jan 2017 18:45:50 +0000 (UTC) (envelope-from Michael.Tuexen@lurchi.franken.de) Received: from drew.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.franken.de", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1C9241E3A; Wed, 11 Jan 2017 18:45:49 +0000 (UTC) (envelope-from Michael.Tuexen@lurchi.franken.de) Received: from [IPv6:2003:cd:6bc9:5900:6d10:a969:52d7:de0f] (p200300CD6BC959006D10A96952D7DE0F.dip0.t-ipconnect.de [IPv6:2003:cd:6bc9:5900:6d10:a969:52d7:de0f]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTPSA id EBC35721E281A; Wed, 11 Jan 2017 19:45:39 +0100 (CET) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 10.2 \(3259\)) Subject: Re: HEADS-UP: IFLIB implementations of sys/dev/e1000 em, lem, igb pending From: Michael Tuexen In-Reply-To: <1eb88092-d9de-e692-4b80-847bd6b71d0f@freebsd.org> Date: Wed, 11 Jan 2017 19:45:39 +0100 Cc: freebsd-net@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <9A868536-08B0-4F56-B12D-720AD955E8DA@lurchi.franken.de> References: <092ad9f7-b04c-292f-c626-6ce1956580a8@freebsd.org> <44D9E8B2-1045-4B01-9EB5-B2F25932D8FB@lurchi.franken.de> <1eb88092-d9de-e692-4b80-847bd6b71d0f@freebsd.org> To: Sean Bruno X-Mailer: Apple Mail (2.3259) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=disabled version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mail-n.franken.de X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jan 2017 18:45:50 -0000 > On 11 Jan 2017, at 18:44, Sean Bruno wrote: >=20 >=20 >=20 > On 01/11/17 10:24, Michael Tuexen wrote: >>> On 10 Jan 2017, at 04:32, Sean Bruno wrote: >>>=20 >>> tl;dir --> you get to keep your igbX devices(thanks jhb), no POLA >>> violations this week. >>>=20 >>> I've updated sys/dev/e1000 at svn R311849 to match Matt Macy's work = on >>> IFLIB in the kernel. >>>=20 >>> At this point, the driver deviates from Intel's code dramatically = and >>> you now get to yell directly into the freebsd-net@ megaphone for = things >>> that I may have broken. >> My testing shows that sending SCTP/IPv4, SCTP/IPv6, TCP/IPv6, and = UDP/IPV6 >> is broken since the checksum offload doesn't work. >>=20 >> The following patch fixes the issue: >>=20 >> Index: if_em.h >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> --- if_em.h (revision 311887) >> +++ if_em.h (working copy) >> @@ -330,7 +330,7 @@ >> #define EM_MSIX_LINK 0x01000000 /* For 82574 use */ >> #define ETH_ZLEN 60 >> #define ETH_ADDR_LEN 6 >> -#define CSUM_OFFLOAD 7 /* Offload bits in mbuf flag = */ >> +#define CSUM_OFFLOAD 0xE0F /* Offload bits in mbuf flag = */ >>=20 >> #define IGB_PKTTYPE_MASK 0x0000FFF0 >> #define IGB_DMCTLX_DCFLUSH_DIS 0x80000000 /* Disable DMA Coalesce = Flush */ >>=20 >> The magic number 7 is CSUM_IP | CSUM_IP_UDP | CSUM_IP_TCP, which = means >> that checksum offloading is working for TCP/IPv4 and UDP/IPv4. >> It should be 0xE0F, which is >> CSUM_IP | CSUM_IP_UDP | CSUM_IP_TCP | CSUM_IP_SCTP | CSUM_IP6_UDP | = CSUM_IP6_TCP | CSUM_IP6_SCTP. >>=20 >> I have tested the above patch with an igb card which supports all = these offloads. >>=20 >> Best regards >> Michael >>>=20 >>> man page updates are coming up next. Please let us know if this >>> revision has made things better, worse or none-of-the above on = whatever >>> Intel Gigabit NIC you happen to have lying around. >>>=20 >>> sean >>>=20 >>> On 01/05/17 20:18, Sean Bruno wrote: >>>> tl;dr --> igbX devices will become emX devices >>>>=20 >>>> We're about to commit an update to sys/dev/e1000 that will = implement and >>>> activate IFLIB for em(4), lem(4) & igb(4) and would appreciate all = folks >>>> who can test and poke at the drivers to do so this week. This will = have >>>> some really great changes for performance and standardization that = have >>>> been bouncing around inside of various FreeBSD shops that have been >>>> collaborating with Matt Macy over the last year. >>>>=20 >>>> This will implement multiple queues for certain em(4) devices that = are >>>> capable of such things and add some new sysctl's for you to poke at = in >>>> your monitoring tools. >>>>=20 >>>> Due to limitations of device registration, igbX devices will become = emX >>>> devices. So, you'll need to make a minor update to your rc.conf = and >>>> scripts that manipulate the network devices. >>>>=20 >>>> UPDATING will be bumped to reflect these changes. >>>>=20 >>>> MFC to stable/11 will have a legacy implementation that doesn't use >>>> IFLIB for compatibility reasons. >>>>=20 >>>> A documentation and man page update will follow in the next few = days >>>> explaining how to work with the changed driver. >>>>=20 >>>> sean >>>>=20 >>>> bcc net@ current@ re@ >>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>=20 >>=20 >>=20 >=20 >=20 > Bah, last one was wrong bits. Try this one. >=20 > Index: em_txrx.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- em_txrx.c (revision 311875) > +++ em_txrx.c (working copy) > @@ -304,7 +304,7 @@ > if (do_tso) { > i =3D em_tso_setup(sc, pi, &txd_upper, &txd_lower); > tso_desc =3D TRUE; > - } else if (csum_flags & CSUM_OFFLOAD) { > + } else if (csum_flags & EM_CSUM_OFFLOAD) { > i =3D em_transmit_checksum_setup(sc, pi, &txd_upper, = &txd_lower); > } >=20 > Index: if_em.h > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- if_em.h (revision 311875) > +++ if_em.h (working copy) > @@ -330,7 +330,8 @@ > #define EM_MSIX_LINK 0x01000000 /* For 82574 use */ > #define ETH_ZLEN 60 > #define ETH_ADDR_LEN 6 > -#define CSUM_OFFLOAD 7 /* Offload bits in mbuf flag */ > +#define EM_CSUM_OFFLOAD 7 /* Offload bits in mbuf = flag */ > +#define IGB_CSUM_OFFLOAD 0x0E0F /* Offload bits in mbuf flag */ >=20 > #define IGB_PKTTYPE_MASK 0x0000FFF0 > #define IGB_DMCTLX_DCFLUSH_DIS 0x80000000 /* Disable DMA = Coalesce Flush */ > Index: igb_txrx.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- igb_txrx.c (revision 311875) > +++ igb_txrx.c (working copy) > @@ -171,7 +171,7 @@ > */ > if (pi->ipi_mflags & M_VLANTAG) { > vlan_macip_lens |=3D (pi->ipi_vtag << = E1000_ADVTXD_VLAN_SHIFT); > - } else if ((pi->ipi_csum_flags & CSUM_OFFLOAD) =3D=3D 0) { > + } else if ((pi->ipi_csum_flags & IGB_CSUM_OFFLOAD) =3D=3D 0) { > return (0); > } Looks good to me... Best regards Michael > =09 >=20