Date: Sat, 16 Apr 2016 14:55:53 +0000 From: Meny Yossefi <menyy@mellanox.com> To: "owner-freebsd-infiniband@freebsd.orgOn" <owner-freebsd-infiniband@freebsd.orgOn>, "freebsd-infiniband@freebsd.org" <freebsd-infiniband@freebsd.org>, Hans Petter Selasky <hanss@mellanox.com>, Yuval Bason <yuvalba@mellanox.com> Subject: RE: How to force IB cards to come up in IB mode? Message-ID: <DB5PR05MB1765AAD188E07C726CE5C005C3690@DB5PR05MB1765.eurprd05.prod.outlook.com> In-Reply-To: <223ee8f3dd5240779d39bf78c383c50c@AMSPR05MB0580.eurprd05.prod.outlook.com> References: <9C08621D-AAE9-42D0-8079-716CDD9BC5E0@postgresql.org> <223ee8f3dd5240779d39bf78c383c50c@AMSPR05MB0580.eurprd05.prod.outlook.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi Justin,
It seems as the pre-boot configuration didn't work.
loader.conf doesn't work on all sysctl params AFAIK.
Is it a pre-boot only issue?
What is the current sysctl output for those ports?
Have you tried switching to IB after system was loaded?
If I remember correctly, the driver was set for Eth configuration by default.
Of course, this is reversible..
Regards,
Meny Yossefi | SW Manager | FreeBSD Team
Mellanox Technologies Ltd
Work: +972-74-7129121, Cell: +972-52-8379557
________________________________________
From: owner-freebsd-infiniband@freebsd.orgOn<mailto:owner-freebsd-infiniband@freebsd.orgOn> Behalf OfJustin Clift
Sent: Saturday, April 16, 2016 11:15:11 AM (UTC) Monrovia, Reykjavik
To: freebsd-infiniband@freebsd.org<mailto:freebsd-infiniband@freebsd.org>
Subject: How to force IB cards to come up in IB mode?
Hi all,
I'm trying to get a new FreeBSD 10.3-STABLE install to work well with my MHGH28-XTC (dual port ConnectX DDR) cards.
The problem is they're coming up in eth mode, not ib mode, and I can't seem to get that fixed. :(
>From boot:
# ifconfig
mlxen0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=6d07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
ether 00:02:c9:00:ca:cd
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
media: Ethernet autoselect
status: no carrier
mlxen1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=6d07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
ether 00:02:c9:00:ca:ce
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
media: Ethernet autoselect
status: no carrier
My loader.conf file seems like it should be correct:
# cat /boot/loader.conf
sys.device.mlx4_core0.mlx4_port1="ib"
sys.device.mlx4_core0.mlx4_port2="ib"
As background, this is a new install (coming from CentOS 6.x, where it all works well), so I know there's no problem with the cards. They're plugged into a 24 port DDR switch (Voltaire ISR 9024D-M) using active optical cables.
Any ideas?
Regards and best wishes,
Justin Clift
--
"My grandfather once told me that there are two kinds of people: those who work and those who take the credit. He told me to try to be in the first group; there was less competition there."
- Indira Gandhi
_______________________________________________
freebsd-infiniband@freebsd.org<mailto:freebsd-infiniband@freebsd.org> mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-infiniband
To unsubscribe, send any mail to "freebsd-infiniband-unsubscribe@freebsd.org<mailto:freebsd-infiniband-unsubscribe@freebsd.org>"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?DB5PR05MB1765AAD188E07C726CE5C005C3690>