From owner-freebsd-infiniband@freebsd.org  Sat Apr 16 11:15:17 2016
Return-Path: <owner-freebsd-infiniband@freebsd.org>
Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1AA13B1043F
 for <freebsd-infiniband@mailman.ysv.freebsd.org>;
 Sat, 16 Apr 2016 11:15:17 +0000 (UTC)
 (envelope-from justin@postgresql.org)
Received: from meldrar.postgresql.org (meldrar.postgresql.org
 [IPv6:2a02:c0:301:0:ffff::31])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits))
 (Client CN "*.postgresql.org", Issuer "Gandi Standard SSL CA 2" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id D643012DC
 for <freebsd-infiniband@freebsd.org>; Sat, 16 Apr 2016 11:15:16 +0000 (UTC)
 (envelope-from justin@postgresql.org)
Received: from 82-69-92-196.dsl.in-addr.zen.co.uk ([82.69.92.196]
 helo=[172.16.1.14]) by meldrar.postgresql.org with esmtpsa
 (TLS1.0:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.84_2)
 (envelope-from <justin@postgresql.org>) id 1arOBy-0001bd-Te
 for freebsd-infiniband@freebsd.org; Sat, 16 Apr 2016 11:15:13 +0000
From: Justin Clift <justin@postgresql.org>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: quoted-printable
Subject: How to force IB cards to come up in IB mode?
Message-Id: <9C08621D-AAE9-42D0-8079-716CDD9BC5E0@postgresql.org>
Date: Sat, 16 Apr 2016 12:15:11 +0100
To: freebsd-infiniband@freebsd.org
Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\))
X-Mailer: Apple Mail (2.1878.6)
X-Pg-Spam-Score: -2.9 (--)
X-BeenThere: freebsd-infiniband@freebsd.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: Infiniband on FreeBSD <freebsd-infiniband.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-infiniband>, 
 <mailto:freebsd-infiniband-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-infiniband/>
List-Post: <mailto:freebsd-infiniband@freebsd.org>
List-Help: <mailto:freebsd-infiniband-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-infiniband>, 
 <mailto:freebsd-infiniband-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 16 Apr 2016 11:15:17 -0000

Hi all,

I'm trying to get a new FreeBSD 10.3-STABLE install to work well with
my MHGH28-XTC (dual port ConnectX DDR) cards.

The problem is they're coming up in eth mode, not ib mode, and I can't
seem to get that fixed. :(

=46rom boot:

  # ifconfig
  mlxen0: flags=3D8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
          =
options=3D6d07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCS=
UM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV=
6>
          ether 00:02:c9:00:ca:cd
          nd6 options=3D29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
          media: Ethernet autoselect
          status: no carrier
  mlxen1: flags=3D8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
          =
options=3D6d07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCS=
UM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV=
6>
          ether 00:02:c9:00:ca:ce
          nd6 options=3D29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
          media: Ethernet autoselect
          status: no carrier

My loader.conf file seems like it should be correct:

  # cat /boot/loader.conf
  sys.device.mlx4_core0.mlx4_port1=3D"ib"
  sys.device.mlx4_core0.mlx4_port2=3D"ib"

As background, this is a new install (coming from CentOS 6.x, where
it all works well), so I know there's no problem with the cards.  =
They're
plugged into a 24 port DDR switch (Voltaire ISR 9024D-M) using active
optical cables.

Any ideas?

Regards and best wishes,

Justin Clift

--
"My grandfather once told me that there are two kinds of people: those
who work and those who take the credit. He told me to try to be in the
first group; there was less competition there."
- Indira Gandhi