From owner-freebsd-net@freebsd.org Mon Feb 10 20:56:43 2020 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 49B8B243F2C for ; Mon, 10 Feb 2020 20:56:43 +0000 (UTC) (envelope-from adam.e.peace@gmail.com) Received: from mail-wm1-x344.google.com (mail-wm1-x344.google.com [IPv6:2a00:1450:4864:20::344]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48GdVy48Qnz42mS; Mon, 10 Feb 2020 20:56:42 +0000 (UTC) (envelope-from adam.e.peace@gmail.com) Received: by mail-wm1-x344.google.com with SMTP id p17so821388wma.1; Mon, 10 Feb 2020 12:56:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=oY5NRWFIGwWU+9j+UaWA5/QEmqc949+sSVdo84RlnGg=; b=OoP/NCN4P4o3O1eY+ikgrFzfEhs6GRfT7pH5ivstKus9gh3BaJLK4ULhTAVAl8vISx ShP1tNrtGqEbajGbcvs/B4VXoJ6Cg2Kx7FPOXqTWhF58D4WzbjealqY3EswSyKz9SFwF 8tn1BOYIiX22fSPDbIzDzE1UDTYR9Bh8bsSbkqS2o9jsOc8lb6I5NHbZ9i1OvEUv19sT zNemAmNFyYG2gUA29uKlcwBOK5s9jYBGLK4UZqKyr8nicuHkMxM7LytaG8kqBAJgn+KO VoCSQwfxDTg+Q7giT8jOTfg1qqlqunfXkbP8+pvicBGpr2LABVcJngn7zUMXbmMK4aKD pgXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=oY5NRWFIGwWU+9j+UaWA5/QEmqc949+sSVdo84RlnGg=; b=OZUsDsUZ2bKU5nVdRaSf2Fs3tlM5IUQWkfpkdyiqr2AZAgJLRTEYwHvuDTKfm2DDQN RIVXYxj0Apv8noY45abWqtSrhAfQtbY8YKi++H/jZq1AqlE1cYQtsj/aEeMzVIdyzm9D h2IW0S6xZdaNJ9zMwYgESqNN9UE5y2YTvT6d4aNpOM2W27ysJBwaiN9jwCv8CFYWSlyu oaBwwuzyxgs+0kQp5PBBHO4A4qVGe2ZUACOitahJ83k0IjvrJI6sBd4PUm7YWASNmgvC HQXvMDnA1ckRsa5g/kQ8HMGQBVRH7a72gMdMWZ5rre9k/Sia8jLGS+daB7F0Uwf0Ngnw qYtA== X-Gm-Message-State: APjAAAXDsFsJq/WDvXqTveHI56aas4/2Rq6/FqDF2AtvymfG4dmER7Ow xaR6STpmEycmV6njIAEeIU8MhRfOTuAloVlOzCgiY0iq X-Google-Smtp-Source: APXvYqwSoULunPZ1Kq22Sm+C47MFpWrdUpoMtN8gSmAnYuhVJGVZg5Nkc0lgy8FxVIaApSQyOt4f1Oy9CCcRxtVysPg= X-Received: by 2002:a1c:988c:: with SMTP id a134mr777013wme.163.1581368200796; Mon, 10 Feb 2020 12:56:40 -0800 (PST) MIME-Version: 1.0 References: <20200207214616.GT4808@kib.kiev.ua> In-Reply-To: <20200207214616.GT4808@kib.kiev.ua> From: Adam Peace Date: Mon, 10 Feb 2020 15:56:29 -0500 Message-ID: Subject: Re: mlx5en - PTYS reg - eth_proto_oper bit 19 not defined To: Konstantin Belousov Cc: freebsd-net@freebsd.org, freebsd-drivers@mellanox.com X-Rspamd-Queue-Id: 48GdVy48Qnz42mS X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=OoP/NCN4; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of adamepeace@gmail.com designates 2a00:1450:4864:20::344 as permitted sender) smtp.mailfrom=adamepeace@gmail.com X-Spamd-Result: default: False [-2.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.996,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_DN_SOME(0.00)[]; URI_COUNT_ODD(1.00)[3]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[4.4.3.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; IP_SCORE(0.00)[ip: (2.63), ipnet: 2a00:1450::/32(-2.47), asn: 15169(-1.72), country: US(-0.05)]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; TAGGED_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Feb 2020 20:56:43 -0000 Thank you for the information. After searching, I found the 50GBase-KR4 link mode is described in this patent: https://patents.google.com/patent/US8891561. I have applied the below patch to my local kernel and the driver appears to be loading and operating properly (without much testing yet). Is there anything else required beyond this? I was unsure of the methodology for defining a value for a new media type. Others seem to have been added sequentially and assigned the next available integer. Is there another method for doing this that I overlooked? Also, how were you able to tell me that the PTYS register eth_proto_oper value of 0x80000 corresponds to 50GBase-KR4? Is there a mapping of these values maintained somewhere? Thanks again for your help. >From 4fa642e98b13628660e3612f0f75e42b042c499b Mon Sep 17 00:00:00 2001 From: Adam Peace <> Date: Mon, 10 Feb 2020 15:17:15 -0500 Subject: [PATCH] Support 50GBase-KR4 media type in mlx5en driver --- sys/dev/mlx5/mlx5_en/mlx5_en_main.c | 4 ++++ sys/dev/mlx5/mlx5_ib/mlx5_ib_main.c | 2 ++ sys/dev/mlx5/port.h | 1 + sys/net/ieee8023ad_lacp.c | 1 + sys/net/if_media.h | 3 +++ 5 files changed, 11 insertions(+) diff --git a/sys/dev/mlx5/mlx5_en/mlx5_en_main.c b/sys/dev/mlx5/mlx5_en/mlx5_en_main.c index 504f6c01591..f6803bc52da 100644 --- a/sys/dev/mlx5/mlx5_en/mlx5_en_main.c +++ b/sys/dev/mlx5/mlx5_en/mlx5_en_main.c @@ -169,6 +169,10 @@ static const struct media mlx5e_mode_table[MLX5E_LINK_SPEEDS_NUMBER][MLX5E_LINK_ .subtype = IFM_50G_KR2, .baudrate = IF_Gbps(50ULL), }, + [MLX5E_50GBASE_KR4][MLX5E_KR4] = { + .subtype = IFM_50G_KR4, + .baudrate = IF_Gbps(50ULL), + }, }; static const struct media mlx5e_ext_mode_table[MLX5E_EXT_LINK_SPEEDS_NUMBER][MLX5E_LINK_MODES_NUMBER] = { diff --git a/sys/dev/mlx5/mlx5_ib/mlx5_ib_main.c b/sys/dev/mlx5/mlx5_ib/mlx5_ib_main.c index eec585eeeb7..38191c1fb77 100644 --- a/sys/dev/mlx5/mlx5_ib/mlx5_ib_main.c +++ b/sys/dev/mlx5/mlx5_ib/mlx5_ib_main.c @@ -166,6 +166,8 @@ static struct net_device *mlx5_ib_get_netdev(struct ib_device *device, static int translate_eth_proto_oper(u32 eth_proto_oper, u8 *active_speed, u8 *active_width) { + /* TODO: Add a case for MLX5E_50GBASE_KR4 where appropriate + */ switch (eth_proto_oper) { case MLX5E_PROT_MASK(MLX5E_1000BASE_CX_SGMII): case MLX5E_PROT_MASK(MLX5E_1000BASE_KX): diff --git a/sys/dev/mlx5/port.h b/sys/dev/mlx5/port.h index 043c7038830..d46ae130349 100644 --- a/sys/dev/mlx5/port.h +++ b/sys/dev/mlx5/port.h @@ -95,6 +95,7 @@ enum mlx5e_link_speed { MLX5E_40GBASE_SR4 = 15, MLX5E_40GBASE_LR4_ER4 = 16, MLX5E_50GBASE_SR2 = 18, + MLX5E_50GBASE_KR4 = 19, MLX5E_100GBASE_CR4 = 20, MLX5E_100GBASE_SR4 = 21, MLX5E_100GBASE_KR4 = 22, diff --git a/sys/net/ieee8023ad_lacp.c b/sys/net/ieee8023ad_lacp.c index 7358b7cfa5e..c5e4125d1d8 100644 --- a/sys/net/ieee8023ad_lacp.c +++ b/sys/net/ieee8023ad_lacp.c @@ -1194,6 +1194,7 @@ lacp_compose_key(struct lacp_port *lp) case IFM_50G_PCIE: case IFM_50G_CR2: case IFM_50G_KR2: + case IFM_50G_KR4: case IFM_50G_SR2: case IFM_50G_LR2: case IFM_50G_LAUI2_AC: diff --git a/sys/net/if_media.h b/sys/net/if_media.h index 8192a6790a0..dad2555e6bd 100644 --- a/sys/net/if_media.h +++ b/sys/net/if_media.h @@ -258,6 +258,7 @@ uint64_t ifmedia_baudrate(int); #define IFM_400G_DR4 IFM_X(115) /* 400GBase-DR4 */ #define IFM_400G_AUI8_AC IFM_X(116) /* 400G-AUI8 active copper/optical */ #define IFM_400G_AUI8 IFM_X(117) /* 400G-AUI8 */ +#define IFM_50G_KR4 IFM_X(118) /* 50GBase-KR4 */ /* * Please update ieee8023ad_lacp.c:lacp_compose_key() @@ -484,6 +485,7 @@ struct ifmedia_description { { IFM_25G_SR, "25GBase-SR" }, \ { IFM_50G_CR2, "50GBase-CR2" }, \ { IFM_50G_KR2, "50GBase-KR2" }, \ + { IFM_50G_KR4, "50GBase-KR4" }, \ { IFM_25G_LR, "25GBase-LR" }, \ { IFM_10G_AOC, "10GBase-AOC" }, \ { IFM_25G_ACC, "25GBase-ACC" }, \ @@ -827,6 +829,7 @@ struct ifmedia_baudrate { { IFM_ETHER | IFM_25G_SR, IF_Gbps(25ULL) }, \ { IFM_ETHER | IFM_50G_CR2, IF_Gbps(50ULL) }, \ { IFM_ETHER | IFM_50G_KR2, IF_Gbps(50ULL) }, \ + { IFM_ETHER | IFM_50G_KR4, IF_Gbps(50ULL) }, \ { IFM_ETHER | IFM_25G_LR, IF_Gbps(25ULL) }, \ { IFM_ETHER | IFM_10G_AOC, IF_Gbps(10ULL) }, \ { IFM_ETHER | IFM_25G_ACC, IF_Gbps(25ULL) }, \ -- 2.22.0 On Fri, Feb 7, 2020 at 4:46 PM Konstantin Belousov wrote: > On Fri, Feb 07, 2020 at 02:15:53PM -0500, Adam Peace wrote: > > Hello, > > > > I'm using the mlx5en driver that's in the FreeBSD 11.3 tree for a > Mellanox > > ConnectX-4 ethernet card on an Azure VM. > > > > The problem that I'm seeing is that the value being retrieved from the > PTYS > > register for eth_proto_oper is 0x80000, which is bit 19 of the field, > but a > > value for bit 19 is not defined in the mlx5e_link_speed enum. This causes > It is 50GBase-KR4 whatever this means. > > > the driver to not find an entry in the mlx5e_mode_table and the > > media.subtype and media.baudrate are not set. This does not stop the > > mlx5en.ko module from loading, but network connectivity is affected. > > > > It seems that this enum is phased out in the upstream Linux driver, but > > it's not clear to me how the replacement method works. Is this a known > > issue? It seems that FreeBSD 12.1 and master still use the method that's > > used in 11.3. Does anyone have any insight on this? > > The problem seems to be that net/if_media.h does not pre-define this mode. > It is somewhat larger patch than just adding the entry in the table in the > driver. >