From owner-freebsd-net@freebsd.org Wed Jan 11 17:44:34 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AAC8CCAB6D9 for ; Wed, 11 Jan 2017 17:44:34 +0000 (UTC) (envelope-from sbruno@freebsd.org) Received: from mail.ignoranthack.me (ignoranthack.me [199.102.79.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7C7B21CF6 for ; Wed, 11 Jan 2017 17:44:33 +0000 (UTC) (envelope-from sbruno@freebsd.org) Received: from [192.168.0.6] (67-0-248-244.albq.qwest.net [67.0.248.244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: sbruno@ignoranthack.me) by mail.ignoranthack.me (Postfix) with ESMTPSA id 94D931928BA; Wed, 11 Jan 2017 17:44:32 +0000 (UTC) Subject: Re: HEADS-UP: IFLIB implementations of sys/dev/e1000 em, lem, igb pending To: Michael Tuexen References: <092ad9f7-b04c-292f-c626-6ce1956580a8@freebsd.org> <44D9E8B2-1045-4B01-9EB5-B2F25932D8FB@lurchi.franken.de> Cc: freebsd-net@freebsd.org From: Sean Bruno Message-ID: <1eb88092-d9de-e692-4b80-847bd6b71d0f@freebsd.org> Date: Wed, 11 Jan 2017 10:44:31 -0700 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.6.0 MIME-Version: 1.0 In-Reply-To: <44D9E8B2-1045-4B01-9EB5-B2F25932D8FB@lurchi.franken.de> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="mi3sCqLqA9HimgkaekCcl67dOH4AjefSu" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jan 2017 17:44:34 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --mi3sCqLqA9HimgkaekCcl67dOH4AjefSu Content-Type: multipart/mixed; boundary="aQVcaUCMiGsQ86WqvuDGf2nsqrAaXnVbo"; protected-headers="v1" From: Sean Bruno To: Michael Tuexen Cc: freebsd-net@freebsd.org Message-ID: <1eb88092-d9de-e692-4b80-847bd6b71d0f@freebsd.org> Subject: Re: HEADS-UP: IFLIB implementations of sys/dev/e1000 em, lem, igb pending References: <092ad9f7-b04c-292f-c626-6ce1956580a8@freebsd.org> <44D9E8B2-1045-4B01-9EB5-B2F25932D8FB@lurchi.franken.de> In-Reply-To: <44D9E8B2-1045-4B01-9EB5-B2F25932D8FB@lurchi.franken.de> --aQVcaUCMiGsQ86WqvuDGf2nsqrAaXnVbo Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 01/11/17 10:24, Michael Tuexen wrote: >> On 10 Jan 2017, at 04:32, Sean Bruno wrote: >> >> tl;dir --> you get to keep your igbX devices(thanks jhb), no POLA >> violations this week. >> >> I've updated sys/dev/e1000 at svn R311849 to match Matt Macy's work on= >> IFLIB in the kernel. >> >> At this point, the driver deviates from Intel's code dramatically and >> you now get to yell directly into the freebsd-net@ megaphone for thing= s >> that I may have broken. > My testing shows that sending SCTP/IPv4, SCTP/IPv6, TCP/IPv6, and UDP/I= PV6 > is broken since the checksum offload doesn't work. >=20 > The following patch fixes the issue: >=20 > Index: if_em.h > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- if_em.h (revision 311887) > +++ if_em.h (working copy) > @@ -330,7 +330,7 @@ > #define EM_MSIX_LINK 0x01000000 /* For 82574 use */ > #define ETH_ZLEN 60 > #define ETH_ADDR_LEN 6 > -#define CSUM_OFFLOAD 7 /* Offload bits in mbuf flag */= > +#define CSUM_OFFLOAD 0xE0F /* Offload bits in mbuf flag */= >=20 > #define IGB_PKTTYPE_MASK 0x0000FFF0 > #define IGB_DMCTLX_DCFLUSH_DIS 0x80000000 /* Disable DMA Coalesce Flu= sh */ >=20 > The magic number 7 is CSUM_IP | CSUM_IP_UDP | CSUM_IP_TCP, which means > that checksum offloading is working for TCP/IPv4 and UDP/IPv4. > It should be 0xE0F, which is > CSUM_IP | CSUM_IP_UDP | CSUM_IP_TCP | CSUM_IP_SCTP | CSUM_IP6_UDP | CSU= M_IP6_TCP | CSUM_IP6_SCTP. >=20 > I have tested the above patch with an igb card which supports all these= offloads. >=20 > Best regards > Michael >> >> man page updates are coming up next. Please let us know if this >> revision has made things better, worse or none-of-the above on whateve= r >> Intel Gigabit NIC you happen to have lying around. >> >> sean >> >> On 01/05/17 20:18, Sean Bruno wrote: >>> tl;dr --> igbX devices will become emX devices >>> >>> We're about to commit an update to sys/dev/e1000 that will implement = and >>> activate IFLIB for em(4), lem(4) & igb(4) and would appreciate all fo= lks >>> who can test and poke at the drivers to do so this week. This will h= ave >>> some really great changes for performance and standardization that ha= ve >>> been bouncing around inside of various FreeBSD shops that have been >>> collaborating with Matt Macy over the last year. >>> >>> This will implement multiple queues for certain em(4) devices that ar= e >>> capable of such things and add some new sysctl's for you to poke at i= n >>> your monitoring tools. >>> >>> Due to limitations of device registration, igbX devices will become e= mX >>> devices. So, you'll need to make a minor update to your rc.conf and >>> scripts that manipulate the network devices. >>> >>> UPDATING will be bumped to reflect these changes. >>> >>> MFC to stable/11 will have a legacy implementation that doesn't use >>> IFLIB for compatibility reasons. >>> >>> A documentation and man page update will follow in the next few days >>> explaining how to work with the changed driver. >>> >>> sean >>> >>> bcc net@ current@ re@ >>> >>> >>> >>> >>> >> >=20 >=20 Bah, last one was wrong bits. Try this one. Index: em_txrx.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- em_txrx.c (revision 311875) +++ em_txrx.c (working copy) @@ -304,7 +304,7 @@ if (do_tso) { i =3D em_tso_setup(sc, pi, &txd_upper, &txd_lower); tso_desc =3D TRUE; - } else if (csum_flags & CSUM_OFFLOAD) { + } else if (csum_flags & EM_CSUM_OFFLOAD) { i =3D em_transmit_checksum_setup(sc, pi, &txd_upper, &txd_lower); } Index: if_em.h =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- if_em.h (revision 311875) +++ if_em.h (working copy) @@ -330,7 +330,8 @@ #define EM_MSIX_LINK 0x01000000 /* For 82574 use */ #define ETH_ZLEN 60 #define ETH_ADDR_LEN 6 -#define CSUM_OFFLOAD 7 /* Offload bits in mbuf flag */ +#define EM_CSUM_OFFLOAD 7 /* Offload bits in mbuf flag */ +#define IGB_CSUM_OFFLOAD 0x0E0F /* Offload bits in mbuf flag */ #define IGB_PKTTYPE_MASK 0x0000FFF0 #define IGB_DMCTLX_DCFLUSH_DIS 0x80000000 /* Disable DMA Coalesce Flush= */ Index: igb_txrx.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- igb_txrx.c (revision 311875) +++ igb_txrx.c (working copy) @@ -171,7 +171,7 @@ */ if (pi->ipi_mflags & M_VLANTAG) { vlan_macip_lens |=3D (pi->ipi_vtag << E1000_ADVTXD_VLAN_SHIFT); - } else if ((pi->ipi_csum_flags & CSUM_OFFLOAD) =3D=3D 0) { + } else if ((pi->ipi_csum_flags & IGB_CSUM_OFFLOAD) =3D=3D 0) { return (0); } =09 --aQVcaUCMiGsQ86WqvuDGf2nsqrAaXnVbo-- --mi3sCqLqA9HimgkaekCcl67dOH4AjefSu Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQGTBAEBCgB9FiEEuq1GMucSHejSCZfdEgHvyh5yfmQFAlh2bwBfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEJB QUQ0NjMyRTcxMjFERThEMjA5OTdERDEyMDFFRkNBMUU3MjdFNjQACgkQEgHvyh5y fmSSTQf/bk97GWQKgJqDEKGDG1tRCWVWr1nLPZfHrorVyLN0LbGj2THz11NJTYx9 VyVCPLqtP+7SsF/gbuXxYbBgydQl0zMdImZWX4Xj9g0aQ7l4e4FsXd7v/rSPa71u eQt3uz3an7NWYD5uSC72wT4MblzdKoinZfWoybS6nSXcVXCetCJiT4qFt+2ckdaP yBEoFz9K/V4Vtwd8gwvu0QikIKTZ3DkAYYbGbSiYVbxOmaSY++VyKQTjsh8z5Q2N bHwNPBbiF/nR7lILoy8tTiWccRhTY8//gtyfc4TW+n2fXHC7ajXZy9DVfToGA7Hu CbHcobJFwvVeZTEdPQERsnTyW4MxPg== =QANH -----END PGP SIGNATURE----- --mi3sCqLqA9HimgkaekCcl67dOH4AjefSu--