From owner-freebsd-net@freebsd.org Thu May 10 11:56:26 2018 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5723DFC640A for ; Thu, 10 May 2018 11:56:26 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from mail.in-addr.com (mail.in-addr.com [IPv6:2a01:4f8:191:61e8::2525:2525]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E81C96BFCA for ; Thu, 10 May 2018 11:56:25 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from gjp by mail.in-addr.com with local (Exim 4.91 (FreeBSD)) (envelope-from ) id 1fGkBM-000MaT-Q1; Thu, 10 May 2018 12:56:24 +0100 Date: Thu, 10 May 2018 12:56:24 +0100 From: Gary Palmer To: Grzegorz Junka Cc: freebsd-net@freebsd.org, justin@postgresql.org Subject: Re: Unrecognized Inifiniband HCA Message-ID: <20180510115624.GC13355@in-addr.com> References: <169976f0-2159-9ef0-d956-b13e717ef783@gjunka.com> <7218e2c4b23ebf4553a352633fe49e72@postgresql.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: gpalmer@freebsd.org X-SA-Exim-Scanned: No (on mail.in-addr.com); SAEximRunCond expanded to false X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 May 2018 11:56:26 -0000 On Thu, May 10, 2018 at 09:21:10AM +0000, Grzegorz Junka wrote: > > On 08/05/2018 23:41, Justin Clift wrote: > > On 2018-05-08 21:59, Grzegorz Junka wrote: > >> Hi All, > >> > >> pciconf -lv > >> > >> gives me > >> > >> none1@pci0:3:0:0:???????????? class=0x0c0600 card=0x000315b3 chip=0x634015b3 > >> rev=0xa0 hdr=0x00 > >> ?????? vendor???????? = 'Mellanox Technologies' > >> ?????? device???????? = 'MT25408 [ConnectX VPI - IB SDR / 10GigE]' > >> ?????? class?????????? = serial bus > >> > >> Does it mean that my card is unrecognized? It supposed to be 10GB x 2 > >> Infiniband PCI-E HCA 500EX-D Dual-Port Card Mellanox Firmware. > >> Currently the card doesn't show when doing ifconfig. What should I do > >> to have the proper device name instead of none1 or for the card to > >> appear in ifconfig? > >> > >> # kldstat > >> Id Refs Address?????????????????????? Size???????? Name > >> ??1???? 28 0xffffffff80200000 1f67a88?? kernel > >> ??2?????? 1 0xffffffff82169000 316708???? zfs.ko > >> ??3?????? 2 0xffffffff82480000 cb78???????? opensolaris.ko > >> ??4?????? 1 0xffffffff8248d000 42c28?????? mps.ko > >> ??5?????? 1 0xffffffff82621000 bdb0???????? if_lagg.ko > >> ??6?????? 1 0xffffffff8262d000 3650???????? ums.ko > >> ??7?????? 1 0xffffffff82631000 6679???????? nullfs.ko > >> ??8?????? 1 0xffffffff82638000 bdfe???????? unionfs.ko > >> ??9?????? 2 0xffffffff82644000 2094f?????? mlx5.ko > >> 10?????? 2 0xffffffff82665000 103e1?????? linuxkpi.ko > >> 11?????? 1 0xffffffff82676000 15965?????? mlx5en.ko > > > > That's probably a ConnectX (series 1) Mellanox card.?? Those can > > operate in > > either Infiniband mode, or Ethernet mode. > > > > Which mode are you wanting it to run in? :) > > > > As a thought, the FreeBSD wiki page has a bit of info: > > > > ?? https://wiki.freebsd.org/InfiniBand > > > > For that card to be recognised at all, it'll need the mlx4 driver(s) > > to load. > > > > I don't remember the exact one off hand (it's been a while), but some > > searching > > online for mlx4 and FreeBSD should turn up the right bits. > > Many thanks Justin. This is the first time I am hearing about an > Infiniband card operating in Ethernet mode. These cards come with two > CX4/SFF 8470 ports. It's not possible to connect standard Ethernet > cables that I know of (not even SFP modules). Do you mean that they can > operate in Ethernet mode over the CX4 cable? You can get CX4 to SFP+ cables, so I would assume so Regards Gary