Date: Mon, 5 Dec 2016 22:53:08 +0000 (GMT) From: Mellanox Support Admin <supportadmin@mellanox.com> To: "freebsd-infiniband@freebsd.org" <freebsd-infiniband@freebsd.org> Subject: Mellanox Case # 00320058 : RE: Slow speed with Mellanox IS5023 Switch and Connect-X3 cards was updated [ref:_00D507heg._50050q9Npg:ref ] Message-ID: <RY-m9000000000000000000000000000000000000000000000OHQI8K007JIpv2FZQLmu7orPmOP7ow@sfdc.net>
next in thread | raw e-mail | index | archive | help
Dear Vin=C3=ADcius Ferr=C3=A3o,=20 Your Mellanox support case # 00320058/ has been updated By , with the fol= lowing comment:=20 /------------------------------------------------------------- Hi Vin=C3=ADcius, You are not using Mellanox OFED Driver if you are running OFED (OpenFabrics= Enterprise Distribution), what is the version and did you consult the UM? With our driver, you can change the mode with: echo datagram > /sys/class/net/ib0/mode sets the mode of ib0 to UD echo connected > /sys/class/net/ib0/mode sets the mode ib0 to CM Though FreeBSD does not have a /sys. Regards, Sophie. -------------------------------------------------------------/ Thank You,=20 =20 Click on the link to access the case:=20 https://mymellanox.force.com/support/5005000000q9Npg Don=E2=80=99t forget to check out our Online Academy, register now to learn= about the products and technology, http://academy.mellanox.com/login/index= .php=20 /------ Your Case Comments History ------=20 ------------------------------------------------- Created by: Sophie Naudin Pruitt 2016-12-05 22:53:06 Hi Vin=C3=ADcius, You are not using Mellanox OFED Driver if you are running OFED (OpenFabrics= Enterprise Distribution), what is the version and did you consult the UM? With our driver, you can change the mode with: echo datagram > /sys/class/net/ib0/mode sets the mode of ib0 to UD echo connected > /sys/class/net/ib0/mode sets the mode ib0 to CM Though FreeBSD does not have a /sys. Regards, Sophie. ------------------------------------------------- Created by: Anderson Dias Silva 2016-12-05 20:29:35 Hi Benny, I am Anderson Dias Silva (LATAM SE). Please, see below an important conside= ration of our customer. ########################################### After some testing we discovered that was a mismatch between the client and= the server on operating modes of IPoIB. One side was using Connected Mode = (the FreeBSD one) and the Linux client was on Datagram mode). So I=E2=80=99m considering running FreeBSD in UD (Datagram) mode to see if = I finally can get rid of the problems. But, how can I change between CM and= UD in FreeBSD? We don't have a /sys folder on FreeBSD so I cant just set f= rom 0 to 1 and change modes. ########################################### So, how can we change between CM and UD in FreeBSD? Ragards! ------------------------------------------------- Created by: Benny Filin 2016-12-05 01:03:55 Hello Vin=C3=ADcius Ferr=C3=A3o, Thank you for contacting Mellanox Technical Support. 1) DId you try changing switch ports when testing performance between two F= reeBSD machines through switch? 2) What happens when you try a back-to-back connection between two FreeBSD = machines? Do you get good results? 3) Did you test the performance between FreeBSD and a Linux machine through= switch? Please, follow the provided procedure in order to query your unmanaged swit= ch:=20 =20 1) Download and install MFT (Mellanox Firmware Tool): http://www.mellanox.com/content/pages.php?pg=3Dmanagement_tools&menu_sectio= n=3D34 Refer to the User Manual for the installation instructions: http://www.mellanox.com/pdf/MFT/MFT_user_manual.pdf =20 2) From one of the nodes in the fabric, invoke: ibswitches. Match the relevant switch GUID from the Mellanox switch label to the LID. =20 3) From one of the nodes in the fabric , invoke: mst start -> mst ib add ->= mst status =20 4) Identify the relevant unmanaged switch mst representation based on the s= witch LID (from previous step 2). * Matching the remote switch mst device representation to the relevant swit= ch procedure: Locate the switch in the "ibswitches" output and retrieve the switch LID.= =20 Convert the LID from Decimal to Hexadecimal base. Locate the device with this LID (in hex base) from the "mst status" output.= =20 5) Invoke the following command, and provide the output: flint -d <remote_switch_mst_device> q We're looking forward to hearing from you. Best regards, Mellanox Technical Support ------------------------------------------------- Created by: Jubran Jubran 2016-12-01 16:41:35 Hello Vin=C3=ADcius Ferr=C3=A3o, Thank you for contacting Mellanox Technical Support. 1) DId you try changing switch ports when testing performance between two F= reeBSD machines through switch? 2) What happens when you try a back-to-back connection between two FreeBSD = machines? Do you get good results? 3) Did you test the performance between FreeBSD and a Linux machine through= switch? Please, follow the provided procedure in order to query your unmanaged swit= ch:=20 =C2=A0 1) Download and install MFT (Mellanox Firmware Tool): http://www.mellanox.com/content/pages.php?pg=3Dmanagement_tools&menu_sectio= n=3D34 Refer to the User Manual for the installation instructions: http://www.mellanox.com/pdf/MFT/MFT_user_manual.pdf =C2=A0 2) From one of the nodes in the fabric, invoke: ibswitches. Match the relevant switch GUID from the Mellanox switch label to the LID. =C2=A0 3) From one of the nodes in the fabric , invoke: mst start -> mst ib add ->= mst status =C2=A0 4) Identify the relevant unmanaged switch mst representation based on the s= witch LID (from previous step 2). * Matching the remote switch mst device representation to the relevant swit= ch procedure: Locate the switch in the "ibswitches" output and retrieve the switch LID.= =20 Convert the LID from Decimal to Hexadecimal base. Locate the device with this LID (in hex base) from the "mst status" output.= =C2=A0 5) Invoke the following command, and provide the output: flint -d <remote_switch_mst_device> q We're looking forward to hearing from you. Best regards, Mellanox Technical Support=20 -------------------------------------------------------/
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?RY-m9000000000000000000000000000000000000000000000OHQI8K007JIpv2FZQLmu7orPmOP7ow>