From owner-svn-src-all@freebsd.org Tue Sep 3 14:07:49 2019 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 05164DD230; Tue, 3 Sep 2019 14:06:58 +0000 (UTC) (envelope-from yuripv@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46N80146QPz4Q5H; Tue, 3 Sep 2019 14:06:57 +0000 (UTC) (envelope-from yuripv@freebsd.org) Received: by freefall.freebsd.org (Postfix, from userid 1452) id 287F21AE2D; Tue, 3 Sep 2019 14:06:23 +0000 (UTC) X-Original-To: yuripv@localmail.freebsd.org Delivered-To: yuripv@localmail.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client CN "mx1.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by freefall.freebsd.org (Postfix) with ESMTPS id 12D9F1F632; Tue, 16 Apr 2019 16:33:43 +0000 (UTC) (envelope-from owner-src-committers@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 411876D5F1; Tue, 16 Apr 2019 16:33:42 +0000 (UTC) (envelope-from owner-src-committers@freebsd.org) Received: by freefall.freebsd.org (Postfix, from userid 538) id 196B81F5C9; Tue, 16 Apr 2019 16:33:42 +0000 (UTC) Delivered-To: src-committers@localmail.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client CN "mx1.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by freefall.freebsd.org (Postfix) with ESMTPS id 2D9DE1F5C1 for ; Tue, 16 Apr 2019 16:33:39 +0000 (UTC) (envelope-from slava.shwartsman@gmail.com) Received: from mail-wm1-f67.google.com (mail-wm1-f67.google.com [209.85.128.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A82D86D5E3; Tue, 16 Apr 2019 16:33:38 +0000 (UTC) (envelope-from slava.shwartsman@gmail.com) Received: by mail-wm1-f67.google.com with SMTP id a184so26208825wma.2; Tue, 16 Apr 2019 09:33:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:subject:to:references:from:message-id :date:user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=xzoyuqJjN97oi8wOmjD+bZ5jJ4Q3yms0vJMbjXOZdek=; b=PAD2i4ni29as0oEejgfsieTUMifZmnrfvl1jA87DraB6lO+pOjX+N30ZzUWnkpQKl6 GAzXVc8PgTp9QNNt+stv6nFl2dhHPby2SHVdnTLI1wyeeujj+u/QY7wkNs4dqnsNumXq Qp0CMrj3oiE24PP33PppIN4VN4GthmIRTbty9mfSiLHb1P5XOxOIRLiRwedFTQK/cXR2 K3TQ0ey1GDmip6nGCO1qdTWNyLU1mbk1cPFc+3ULndM1eiXVNohlYIxCB4aZ3onkn/Qq Jskqk7u+62RWrVoho0X0jx8lWD+tOwlBHiDUXugTA+uGsgSJCI+ry14NJDg77FtgmPR1 LWGA== X-Gm-Message-State: APjAAAUTLq8hEdUYEF0s+hTecR/q7SF7fISGkMAMj0KaBvtT4Q8Ds3+D p6hceKYKYD5pZyC6P1VC73JOnb+r X-Google-Smtp-Source: APXvYqwvGhLoK3jeAtwZQgIsjdXaqrR8XrlT5x1870igppgBbaU+xhaiPi1ZZuVlLAUegrbMwJs+uA== X-Received: by 2002:a1c:ac07:: with SMTP id v7mr2667693wme.49.1555428391734; Tue, 16 Apr 2019 08:26:31 -0700 (PDT) Received: from [10.0.0.30] (bzq-79-181-3-237.red.bezeqint.net. [79.181.3.237]) by smtp.gmail.com with ESMTPSA id g12sm62178942wrw.40.2019.04.16.08.26.30 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 16 Apr 2019 08:26:30 -0700 (PDT) Reply-To: slavash@FreeBSD.org Subject: Re: svn commit: r341586 - head/sys/dev/mlx5/mlx5_en To: "Andrey V. Elsukov" , src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org, slavash@mellanox.com References: <201812051425.wB5EP38T004562@repo.freebsd.org> From: Slava Shwartsman Message-ID: User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Precedence: bulk X-Loop: FreeBSD.org Sender: owner-src-committers@freebsd.org X-Rspamd-Queue-Id: 411876D5F1 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.99 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.99)[-0.987,0]; REPLY(-4.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0] Status: O X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Date: Tue, 03 Sep 2019 14:07:49 -0000 X-Original-Date: Tue, 16 Apr 2019 18:26:28 +0300 X-List-Received-Date: Tue, 03 Sep 2019 14:07:49 -0000 On 16-Apr-19 17:39, Andrey V. Elsukov wrote: > On 05.12.2018 17:25, Slava Shwartsman wrote: >> Author: slavash >> Date: Wed Dec 5 14:25:03 2018 >> New Revision: 341586 >> URL: https://svnweb.freebsd.org/changeset/base/341586 >> >> Log: >> mlx5en: Implement backpressure indication. >> >> The backpressure indication is implemented using an unlimited rate type of >> mbuf send tag. When the upper layers typically the socket layer has obtained such >> a tag, it can then query the destination driver queue for the current >> amount of space available in the send queue. >> >> A single mbuf send tag may be referenced multiple times and a refcount has been added >> to the mlx5e_priv structure to track its usage. Because the send tag resides >> in the mlx5e_channel structure, there is no need to wait for refcounts to reach >> zero until the mlx4en(4) driver is detached. The channels structure is persistant >> during the lifetime of the mlx5en(4) driver it belongs to and can so be accessed >> without any need of synchronization. >> >> The mlx5e_snd_tag structure was extended to contain a type field, because there are now >> two different tag types which end up in the driver which need to be distinguished. >> >> Submitted by: hselasky@ >> Approved by: hselasky (mentor) >> MFC after: 1 week >> Sponsored by: Mellanox Technologies >> @@ -587,27 +609,33 @@ mlx5e_xmit(struct ifnet *ifp, struct mbuf *mb) >> struct mlx5e_sq *sq; >> int ret; >> >> - sq = mlx5e_select_queue(ifp, mb); >> - if (unlikely(sq == NULL)) { >> -#ifdef RATELIMIT >> - /* Check for route change */ >> - if (mb->m_pkthdr.snd_tag != NULL && >> - mb->m_pkthdr.snd_tag->ifp != ifp) { >> + if (mb->m_pkthdr.snd_tag != NULL) { >> + sq = mlx5e_select_queue_by_send_tag(ifp, mb); >> + if (unlikely(sq == NULL)) { >> + /* Check for route change */ >> + if (mb->m_pkthdr.snd_tag->ifp != ifp) { >> + /* Free mbuf */ >> + m_freem(mb); >> + >> + /* >> + * Tell upper layers about route >> + * change and to re-transmit this >> + * packet: >> + */ >> + return (EAGAIN); >> + } > > Hi, > > I just discovered something strange and found that this commit is the > cause. > The test system has mlx5en 100G interface. It has two vlans: vlan500 and > vlan100. > Via vlan500 it receives some packets flows. Then it routes these packets > into vlan100. > But packets are dropped in mlx5e_xmit() with EAGAIN error code. > > # dtrace -n 'fbt::ip6_output:return {printf("%d", arg1);}' > dtrace: description 'fbt::ip6_output:return ' matched 1 probe > CPU ID FUNCTION:NAME > 23 54338 ip6_output:return 35 > 16 54338 ip6_output:return 35 > 21 54338 ip6_output:return 35 > 22 54338 ip6_output:return 35 > 24 54338 ip6_output:return 35 > 23 54338 ip6_output:return 35 > 14 54338 ip6_output:return 35 > ^C > > # dtrace -n 'fbt::mlx5e_xmit:return {printf("%d", arg1);}' > dtrace: description 'fbt::mlx5e_xmit:return ' matched 1 probe > CPU ID FUNCTION:NAME > 16 69030 mlx5e_xmit:return 35 > 23 69030 mlx5e_xmit:return 35 > 26 69030 mlx5e_xmit:return 35 > 25 69030 mlx5e_xmit:return 35 > 24 69030 mlx5e_xmit:return 35 > 21 69030 mlx5e_xmit:return 35 > 26 69030 mlx5e_xmit:return 35 > ^C > > The kernel config is GENERIC. > 13.0-CURRENT #9 r345758+82f3d57(svn_head)-dirty > Hi Andrey, Thanks for letting us know about this regression. I would like to try to reproduce this issue in house. Can you please share the exact steps to reproduce it? - Can I reproduce the issue with B2B setup? - What is the route command you used to make the route between the VLANs? - What app are you using to generate the traffic? Slava