Skip site navigation (1)Skip section navigation (2)
Date:      Tue, 16 Apr 2019 17:39:47 +0300
From:      "Andrey V. Elsukov" <bu7cher@yandex.ru>
To:        Slava Shwartsman <slavash@FreeBSD.org>, src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org
Subject:   Re: svn commit: r341586 - head/sys/dev/mlx5/mlx5_en
Message-ID:  <a1ff0879-abae-25c5-9350-809186d2cf85@yandex.ru>
In-Reply-To: <201812051425.wB5EP38T004562@repo.freebsd.org>
References:  <201812051425.wB5EP38T004562@repo.freebsd.org>

next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--Gc4FCLxF5RLpVDUgQhYnJWt4CbwtXFWML
Content-Type: multipart/mixed; boundary="CkT7XhrlVpnZaZvea4uGJsDAgElFOplO3";
 protected-headers="v1"
From: "Andrey V. Elsukov" <bu7cher@yandex.ru>
To: Slava Shwartsman <slavash@FreeBSD.org>, src-committers@freebsd.org,
 svn-src-all@freebsd.org, svn-src-head@freebsd.org
Message-ID: <a1ff0879-abae-25c5-9350-809186d2cf85@yandex.ru>
Subject: Re: svn commit: r341586 - head/sys/dev/mlx5/mlx5_en
References: <201812051425.wB5EP38T004562@repo.freebsd.org>
In-Reply-To: <201812051425.wB5EP38T004562@repo.freebsd.org>

--CkT7XhrlVpnZaZvea4uGJsDAgElFOplO3
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable

On 05.12.2018 17:25, Slava Shwartsman wrote:
> Author: slavash
> Date: Wed Dec  5 14:25:03 2018
> New Revision: 341586
> URL: https://svnweb.freebsd.org/changeset/base/341586
>=20
> Log:
>   mlx5en: Implement backpressure indication.
>  =20
>   The backpressure indication is implemented using an unlimited rate ty=
pe of
>   mbuf send tag. When the upper layers typically the socket layer has o=
btained such
>   a tag, it can then query the destination driver queue for the current=

>   amount of space available in the send queue.
>  =20
>   A single mbuf send tag may be referenced multiple times and a refcoun=
t has been added
>   to the mlx5e_priv structure to track its usage. Because the send tag =
resides
>   in the mlx5e_channel structure, there is no need to wait for refcount=
s to reach
>   zero until the mlx4en(4) driver is detached. The channels structure i=
s persistant
>   during the lifetime of the mlx5en(4) driver it belongs to and can so =
be accessed
>   without any need of synchronization.
>  =20
>   The mlx5e_snd_tag structure was extended to contain a type field, bec=
ause there are now
>   two different tag types which end up in the driver which need to be d=
istinguished.
>  =20
>   Submitted by:   hselasky@
>   Approved by:    hselasky (mentor)
>   MFC after:      1 week
>   Sponsored by:   Mellanox Technologies
> @@ -587,27 +609,33 @@ mlx5e_xmit(struct ifnet *ifp, struct mbuf *mb)
>  	struct mlx5e_sq *sq;
>  	int ret;
> =20
> -	sq =3D mlx5e_select_queue(ifp, mb);
> -	if (unlikely(sq =3D=3D NULL)) {
> -#ifdef RATELIMIT
> -		/* Check for route change */
> -		if (mb->m_pkthdr.snd_tag !=3D NULL &&
> -		    mb->m_pkthdr.snd_tag->ifp !=3D ifp) {
> +	if (mb->m_pkthdr.snd_tag !=3D NULL) {
> +		sq =3D mlx5e_select_queue_by_send_tag(ifp, mb);
> +		if (unlikely(sq =3D=3D NULL)) {
> +			/* Check for route change */
> +			if (mb->m_pkthdr.snd_tag->ifp !=3D ifp) {
> +				/* Free mbuf */
> +				m_freem(mb);
> +
> +				/*
> +				 * Tell upper layers about route
> +				 * change and to re-transmit this
> +				 * packet:
> +				 */
> +				return (EAGAIN);
> +			}

Hi,

I just discovered something strange and found that this commit is the
cause.
The test system has mlx5en 100G interface. It has two vlans: vlan500 and
vlan100.
Via vlan500 it receives some packets flows. Then it routes these packets
into vlan100.
But packets are dropped in mlx5e_xmit() with EAGAIN error code.

# dtrace -n 'fbt::ip6_output:return {printf("%d", arg1);}'
dtrace: description 'fbt::ip6_output:return ' matched 1 probe
CPU     ID                    FUNCTION:NAME
 23  54338                ip6_output:return 35
 16  54338                ip6_output:return 35
 21  54338                ip6_output:return 35
 22  54338                ip6_output:return 35
 24  54338                ip6_output:return 35
 23  54338                ip6_output:return 35
 14  54338                ip6_output:return 35
^C

# dtrace -n 'fbt::mlx5e_xmit:return {printf("%d", arg1);}'
dtrace: description 'fbt::mlx5e_xmit:return ' matched 1 probe
CPU     ID                    FUNCTION:NAME
 16  69030                mlx5e_xmit:return 35
 23  69030                mlx5e_xmit:return 35
 26  69030                mlx5e_xmit:return 35
 25  69030                mlx5e_xmit:return 35
 24  69030                mlx5e_xmit:return 35
 21  69030                mlx5e_xmit:return 35
 26  69030                mlx5e_xmit:return 35
^C

The kernel config is GENERIC.
13.0-CURRENT #9 r345758+82f3d57(svn_head)-dirty

--=20
WBR, Andrey V. Elsukov


--CkT7XhrlVpnZaZvea4uGJsDAgElFOplO3--

--Gc4FCLxF5RLpVDUgQhYnJWt4CbwtXFWML
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/

iQEzBAEBCAAdFiEE5lkeG0HaFRbwybwAAcXqBBDIoXoFAly16TMACgkQAcXqBBDI
oXoZfAf9Glf7TAruR75X9vsJ2J7jytk8xLTYCTW+U3kmjOToyIWEpmy1HNUDWN7p
jjBvOijhFwZSXzV6mLL/Iy7D3xtomzl/a2Dn4sCKoM5TvYvHim2aQZtPlDnG3BI6
o4Qs705lnCl0li46Ilpmvv9wtdXCXqf9U9ayJe+3BPLojy/ryiXSrdT0EJyI0F3B
d+pyntzQDw1EHlFXrn1paHiTkBjt1V+Hk7gwPP+w8jH6MnIj+Z2Awx6UpGbHfQ36
29JI087K98zG4AIadH0YPj0+myk2eVFzMSQj5Sr/LPU1HwPBtZzLVYD4zEjeRgKk
GnzVIEoAWxkKU4Q3l2FgfM1csAMhWw==
=yi8C
-----END PGP SIGNATURE-----

--Gc4FCLxF5RLpVDUgQhYnJWt4CbwtXFWML--



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?a1ff0879-abae-25c5-9350-809186d2cf85>