From owner-freebsd-net@freebsd.org Thu May 10 11:52:15 2018 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B2ACEFC6137 for ; Thu, 10 May 2018 11:52:15 +0000 (UTC) (envelope-from gjp@in-addr.com) Received: from mail.in-addr.com (mail.in-addr.com [IPv6:2a01:4f8:191:61e8::2525:2525]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4E6556A2F0; Thu, 10 May 2018 11:52:15 +0000 (UTC) (envelope-from gjp@in-addr.com) Received: from gjp by mail.in-addr.com with local (Exim 4.91 (FreeBSD)) (envelope-from ) id 1fGk7I-000MZW-VL; Thu, 10 May 2018 12:52:13 +0100 Date: Thu, 10 May 2018 12:52:12 +0100 From: Gary Palmer To: Grzegorz Junka Cc: freebsd-net@freebsd.org, gpalmer@freebsd.org, freebsd-rwg@pdx.rh.CN85.dnsmgr.net Subject: Re: Missing mlx4, was Re: Unrecognized Inifiniband HCA Message-ID: <20180510115212.GB13355@in-addr.com> References: <201805090039.w490dfXn063341@pdx.rh.CN85.dnsmgr.net> <848a8ac1-0b04-82fe-eb01-e5b7ec331ccb@gjunka.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <848a8ac1-0b04-82fe-eb01-e5b7ec331ccb@gjunka.com> X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: gjp@in-addr.com X-SA-Exim-Scanned: No (on mail.in-addr.com); SAEximRunCond expanded to false X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 May 2018 11:52:16 -0000 On Thu, May 10, 2018 at 08:36:44AM +0000, Grzegorz Junka wrote: > > On 09/05/2018 00:39, Rodney W. Grimes wrote: > >> On Tue, May 08, 2018 at 08:59:20PM +0000, Grzegorz Junka wrote: > >>> Hi All, > >>> > >>> pciconf -lv > >>> > >>> gives me > >>> > >>> none1@pci0:3:0:0:???????????? class=0x0c0600 card=0x000315b3 chip=0x634015b3 > >>> rev=0xa0 hdr=0x00 > >>> ?????? vendor???????? = 'Mellanox Technologies' > >>> ?????? device???????? = 'MT25408 [ConnectX VPI - IB SDR / 10GigE]' > >>> ?????? class?????????? = serial bus > >>> > >>> Does it mean that my card is unrecognized? It supposed to be 10GB x 2 > >>> Infiniband PCI-E HCA 500EX-D Dual-Port Card Mellanox Firmware. Currently > >>> the card doesn't show when doing ifconfig. What should I do to have the > >>> proper device name instead of none1 or for the card to appear in ifconfig? > >>> > >>> # kldstat > >>> Id Refs Address?????????????????????? Size???????? Name > >>> ??1???? 28 0xffffffff80200000 1f67a88?? kernel > >>> ??2?????? 1 0xffffffff82169000 316708???? zfs.ko > >>> ??3?????? 2 0xffffffff82480000 cb78???????? opensolaris.ko > >>> ??4?????? 1 0xffffffff8248d000 42c28?????? mps.ko > >>> ??5?????? 1 0xffffffff82621000 bdb0???????? if_lagg.ko > >>> ??6?????? 1 0xffffffff8262d000 3650???????? ums.ko > >>> ??7?????? 1 0xffffffff82631000 6679???????? nullfs.ko > >>> ??8?????? 1 0xffffffff82638000 bdfe???????? unionfs.ko > >>> ??9?????? 2 0xffffffff82644000 2094f?????? mlx5.ko > >>> 10?????? 2 0xffffffff82665000 103e1?????? linuxkpi.ko > >>> 11?????? 1 0xffffffff82676000 15965?????? mlx5en.ko > >> mlx5en is for ConnectX-4. I think that's an older card. Try mlx4en, > >> which supports ConnectX-2 and ConnectX-3. > > From a quick grep this infact should be supported by the mlx4/mlx5en > > drivers: > > net/mlx4/main.c: /* MT25408 "Hermon" SDR */ > > net/mlx4/main.c: /* MT25408 "Hermon" DDR */ > > net/mlx4/main.c: /* MT25408 "Hermon" QDR */ > > net/mlx4/main.c: /* MT25408 "Hermon" DDR PCIe gen2 */ > > net/mlx4/main.c: /* MT25408 "Hermon" QDR PCIe gen2 */ > > net/mlx4/main.c: /* MT25408 "Hermon" EN 10GigE */ > > net/mlx4/main.c: /* MT25408 "Hermon" EN 10GigE PCIe gen2 */ > > > > Thank you Gary and Rod for your quick reply. I don't seem to have mlx4 > in my /boot/kernel. I unloaded mlx5 and mlx5en and tried to load mlx > instead, but strangely, it says mlx is already loaded. How? > > # kldload mlx > kldload: can't load mlx: module already loaded or in kernel > > # kldstat > Id Refs Address?????????????????????? Size???????? Name > ??1???? 22 0xffffffff80200000 1f67a88?? kernel > ??2?????? 1 0xffffffff82169000 316708???? zfs.ko > ??3?????? 2 0xffffffff82480000 cb78???????? opensolaris.ko > ??4?????? 1 0xffffffff8248d000 42c28?????? mps.ko > ??5?????? 1 0xffffffff82621000 bdb0???????? if_lagg.ko > ??6?????? 1 0xffffffff8262d000 3650???????? ums.ko > ??7?????? 1 0xffffffff82631000 6679???????? nullfs.ko > ??8?????? 1 0xffffffff82638000 bdfe???????? unionfs.ko > > # ls -l /boot/kernel/mlx > mlx.ko*?????? mlx5.ko*???? mlx5en.ko* > > Anyways, mlx seems to be rather unrelated to this problem: > https://www.freebsd.org/cgi/man.cgi?query=mlx > > Looks like mlx4 is available in the kernel: > https://github.com/freebsd/freebsd/tree/master/sys/modules/mlx4 > > However, it seems that it may not compile: > https://forums.freebsd.org/threads/mellanox-mt26448.64350/ > > Does it mean, that mlx4 is no longer compiled in the default kernel and > I would need to compile the kernel manually? Can I just compile the > mlx4/en/ib kernel module without having to compile the whole kernel? You either need to put "options OFED" into your kernel and recompile to get the mlx4 module, or go to /sys/modules/mlx4 and run "make all install" (more details on the wiki that someone else posted earlier) Regards, Gary