From owner-freebsd-infiniband@freebsd.org Tue May 8 21:14:54 2018 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 57B76FCA674; Tue, 8 May 2018 21:14:54 +0000 (UTC) (envelope-from list1@gjunka.com) Received: from msa1.earth.yoonka.com (yoonka.com [88.98.225.149]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "msa1.earth.yoonka.com", Issuer "msa1.earth.yoonka.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id DEA8B6FB4B; Tue, 8 May 2018 21:14:50 +0000 (UTC) (envelope-from list1@gjunka.com) Received: from ultrabook.yoonka.com (p5DD75584.dip0.t-ipconnect.de [93.215.85.132]) (authenticated bits=0) by msa1.earth.yoonka.com (8.15.2/8.15.2) with ESMTPSA id w48KxP1Z009739 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Tue, 8 May 2018 20:59:26 GMT (envelope-from list1@gjunka.com) X-Authentication-Warning: msa1.earth.yoonka.com: Host p5DD75584.dip0.t-ipconnect.de [93.215.85.132] claimed to be ultrabook.yoonka.com To: freebsd-infiniband@freebsd.org, freebsd-net@freebsd.org From: Grzegorz Junka Subject: Unrecognized Inifiniband HCA Message-ID: <169976f0-2159-9ef0-d956-b13e717ef783@gjunka.com> Date: Tue, 8 May 2018 20:59:20 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-GB-large X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 May 2018 21:14:54 -0000 Hi All, pciconf -lv gives me none1@pci0:3:0:0:       class=0x0c0600 card=0x000315b3 chip=0x634015b3 rev=0xa0 hdr=0x00     vendor     = 'Mellanox Technologies'     device     = 'MT25408 [ConnectX VPI - IB SDR / 10GigE]'     class      = serial bus Does it mean that my card is unrecognized? It supposed to be 10GB x 2 Infiniband PCI-E HCA 500EX-D Dual-Port Card Mellanox Firmware. Currently the card doesn't show when doing ifconfig. What should I do to have the proper device name instead of none1 or for the card to appear in ifconfig? # kldstat Id Refs Address            Size     Name  1   28 0xffffffff80200000 1f67a88  kernel  2    1 0xffffffff82169000 316708   zfs.ko  3    2 0xffffffff82480000 cb78     opensolaris.ko  4    1 0xffffffff8248d000 42c28    mps.ko  5    1 0xffffffff82621000 bdb0     if_lagg.ko  6    1 0xffffffff8262d000 3650     ums.ko  7    1 0xffffffff82631000 6679     nullfs.ko  8    1 0xffffffff82638000 bdfe     unionfs.ko  9    2 0xffffffff82644000 2094f    mlx5.ko 10    2 0xffffffff82665000 103e1    linuxkpi.ko 11    1 0xffffffff82676000 15965    mlx5en.ko Thanks GrzegorzJ From owner-freebsd-infiniband@freebsd.org Tue May 8 23:42:03 2018 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EE2E7FCDD14; Tue, 8 May 2018 23:42:02 +0000 (UTC) (envelope-from justin@postgresql.org) Received: from meldrar.postgresql.org (meldrar.postgresql.org [IPv6:2a02:c0:301:0:ffff::31]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "meldrar.postgresql.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8683A72EF1; Tue, 8 May 2018 23:42:02 +0000 (UTC) (envelope-from justin@postgresql.org) Received: from meldrar.postgresql.org ([87.238.57.231] helo=webmail.postgresql.org) by meldrar.postgresql.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.89) (envelope-from ) id 1fGCF2-0006A1-Vy; Tue, 08 May 2018 23:42:00 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Wed, 09 May 2018 00:41:55 +0100 From: Justin Clift To: Grzegorz Junka Cc: freebsd-infiniband@freebsd.org, freebsd-net@freebsd.org, owner-freebsd-infiniband@freebsd.org Subject: Re: Unrecognized Inifiniband HCA In-Reply-To: <169976f0-2159-9ef0-d956-b13e717ef783@gjunka.com> References: <169976f0-2159-9ef0-d956-b13e717ef783@gjunka.com> Message-ID: <7218e2c4b23ebf4553a352633fe49e72@postgresql.org> X-Sender: justin@postgresql.org User-Agent: Roundcube Webmail/1.2.3 X-Pg-Spam-Score: -2.9 (--) X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 May 2018 23:42:03 -0000 On 2018-05-08 21:59, Grzegorz Junka wrote: > Hi All, > > pciconf -lv > > gives me > > none1@pci0:3:0:0:       class=0x0c0600 card=0x000315b3 chip=0x634015b3 > rev=0xa0 hdr=0x00 >     vendor     = 'Mellanox Technologies' >     device     = 'MT25408 [ConnectX VPI - IB SDR / 10GigE]' >     class      = serial bus > > Does it mean that my card is unrecognized? It supposed to be 10GB x 2 > Infiniband PCI-E HCA 500EX-D Dual-Port Card Mellanox Firmware. > Currently the card doesn't show when doing ifconfig. What should I do > to have the proper device name instead of none1 or for the card to > appear in ifconfig? > > # kldstat > Id Refs Address            Size     Name >  1   28 0xffffffff80200000 1f67a88  kernel >  2    1 0xffffffff82169000 316708   zfs.ko >  3    2 0xffffffff82480000 cb78     opensolaris.ko >  4    1 0xffffffff8248d000 42c28    mps.ko >  5    1 0xffffffff82621000 bdb0     if_lagg.ko >  6    1 0xffffffff8262d000 3650     ums.ko >  7    1 0xffffffff82631000 6679     nullfs.ko >  8    1 0xffffffff82638000 bdfe     unionfs.ko >  9    2 0xffffffff82644000 2094f    mlx5.ko > 10    2 0xffffffff82665000 103e1    linuxkpi.ko > 11    1 0xffffffff82676000 15965    mlx5en.ko That's probably a ConnectX (series 1) Mellanox card. Those can operate in either Infiniband mode, or Ethernet mode. Which mode are you wanting it to run in? :) As a thought, the FreeBSD wiki page has a bit of info: https://wiki.freebsd.org/InfiniBand For that card to be recognised at all, it'll need the mlx4 driver(s) to load. I don't remember the exact one off hand (it's been a while), but some searching online for mlx4 and FreeBSD should turn up the right bits. Hope that helps. :) Regards and best wishes, Justin Clift From owner-freebsd-infiniband@freebsd.org Tue May 8 23:42:27 2018 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 95F1FFCDD64; Tue, 8 May 2018 23:42:27 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from mail.in-addr.com (mail.in-addr.com [IPv6:2a01:4f8:191:61e8::2525:2525]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2D30272F97; Tue, 8 May 2018 23:42:27 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from gjp by mail.in-addr.com with local (Exim 4.91 (FreeBSD)) (envelope-from ) id 1fGCFV-0007qS-N8; Wed, 09 May 2018 00:42:25 +0100 Date: Wed, 9 May 2018 00:42:25 +0100 From: Gary Palmer To: Grzegorz Junka Cc: freebsd-infiniband@freebsd.org, freebsd-net@freebsd.org Subject: Re: Unrecognized Inifiniband HCA Message-ID: <20180508234225.GA13355@in-addr.com> References: <169976f0-2159-9ef0-d956-b13e717ef783@gjunka.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <169976f0-2159-9ef0-d956-b13e717ef783@gjunka.com> X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: gpalmer@freebsd.org X-SA-Exim-Scanned: No (on mail.in-addr.com); SAEximRunCond expanded to false X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 May 2018 23:42:27 -0000 On Tue, May 08, 2018 at 08:59:20PM +0000, Grzegorz Junka wrote: > Hi All, > > pciconf -lv > > gives me > > none1@pci0:3:0:0:???????????? class=0x0c0600 card=0x000315b3 chip=0x634015b3 > rev=0xa0 hdr=0x00 > ?????? vendor???????? = 'Mellanox Technologies' > ?????? device???????? = 'MT25408 [ConnectX VPI - IB SDR / 10GigE]' > ?????? class?????????? = serial bus > > Does it mean that my card is unrecognized? It supposed to be 10GB x 2 > Infiniband PCI-E HCA 500EX-D Dual-Port Card Mellanox Firmware. Currently > the card doesn't show when doing ifconfig. What should I do to have the > proper device name instead of none1 or for the card to appear in ifconfig? > > # kldstat > Id Refs Address?????????????????????? Size???????? Name > ??1???? 28 0xffffffff80200000 1f67a88?? kernel > ??2?????? 1 0xffffffff82169000 316708???? zfs.ko > ??3?????? 2 0xffffffff82480000 cb78???????? opensolaris.ko > ??4?????? 1 0xffffffff8248d000 42c28?????? mps.ko > ??5?????? 1 0xffffffff82621000 bdb0???????? if_lagg.ko > ??6?????? 1 0xffffffff8262d000 3650???????? ums.ko > ??7?????? 1 0xffffffff82631000 6679???????? nullfs.ko > ??8?????? 1 0xffffffff82638000 bdfe???????? unionfs.ko > ??9?????? 2 0xffffffff82644000 2094f?????? mlx5.ko > 10?????? 2 0xffffffff82665000 103e1?????? linuxkpi.ko > 11?????? 1 0xffffffff82676000 15965?????? mlx5en.ko mlx5en is for ConnectX-4. I think that's an older card. Try mlx4en, which supports ConnectX-2 and ConnectX-3. Regards, Gary From owner-freebsd-infiniband@freebsd.org Wed May 9 00:39:48 2018 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E1DC0FCF992; Wed, 9 May 2018 00:39:47 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 494167F27E; Wed, 9 May 2018 00:39:46 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w490dgK2063342; Tue, 8 May 2018 17:39:42 -0700 (PDT) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w490dfXn063341; Tue, 8 May 2018 17:39:41 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201805090039.w490dfXn063341@pdx.rh.CN85.dnsmgr.net> Subject: Re: Unrecognized Inifiniband HCA In-Reply-To: <20180508234225.GA13355@in-addr.com> To: Gary Palmer Date: Tue, 8 May 2018 17:39:41 -0700 (PDT) CC: Grzegorz Junka , freebsd-infiniband@freebsd.org, freebsd-net@freebsd.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 May 2018 00:39:48 -0000 > On Tue, May 08, 2018 at 08:59:20PM +0000, Grzegorz Junka wrote: > > Hi All, > > > > pciconf -lv > > > > gives me > > > > none1@pci0:3:0:0:???????????? class=0x0c0600 card=0x000315b3 chip=0x634015b3 > > rev=0xa0 hdr=0x00 > > ?????? vendor???????? = 'Mellanox Technologies' > > ?????? device???????? = 'MT25408 [ConnectX VPI - IB SDR / 10GigE]' > > ?????? class?????????? = serial bus > > > > Does it mean that my card is unrecognized? It supposed to be 10GB x 2 > > Infiniband PCI-E HCA 500EX-D Dual-Port Card Mellanox Firmware. Currently > > the card doesn't show when doing ifconfig. What should I do to have the > > proper device name instead of none1 or for the card to appear in ifconfig? > > > > # kldstat > > Id Refs Address?????????????????????? Size???????? Name > > ??1???? 28 0xffffffff80200000 1f67a88?? kernel > > ??2?????? 1 0xffffffff82169000 316708???? zfs.ko > > ??3?????? 2 0xffffffff82480000 cb78???????? opensolaris.ko > > ??4?????? 1 0xffffffff8248d000 42c28?????? mps.ko > > ??5?????? 1 0xffffffff82621000 bdb0???????? if_lagg.ko > > ??6?????? 1 0xffffffff8262d000 3650???????? ums.ko > > ??7?????? 1 0xffffffff82631000 6679???????? nullfs.ko > > ??8?????? 1 0xffffffff82638000 bdfe???????? unionfs.ko > > ??9?????? 2 0xffffffff82644000 2094f?????? mlx5.ko > > 10?????? 2 0xffffffff82665000 103e1?????? linuxkpi.ko > > 11?????? 1 0xffffffff82676000 15965?????? mlx5en.ko > > mlx5en is for ConnectX-4. I think that's an older card. Try mlx4en, > which supports ConnectX-2 and ConnectX-3. >From a quick grep this infact should be supported by the mlx4/mlx5en drivers: net/mlx4/main.c: /* MT25408 "Hermon" SDR */ net/mlx4/main.c: /* MT25408 "Hermon" DDR */ net/mlx4/main.c: /* MT25408 "Hermon" QDR */ net/mlx4/main.c: /* MT25408 "Hermon" DDR PCIe gen2 */ net/mlx4/main.c: /* MT25408 "Hermon" QDR PCIe gen2 */ net/mlx4/main.c: /* MT25408 "Hermon" EN 10GigE */ net/mlx4/main.c: /* MT25408 "Hermon" EN 10GigE PCIe gen2 */ > > Regards, > > Gary -- Rod Grimes rgrimes@freebsd.org