Skip site navigation (1)Skip section navigation (2) 
Date:      Wed, 07 Jan 2015 14:24:04 +0000
From:      Karl Pielorz <kpielorz_lst@tdx.co.uk>
To:        Oded Shanoon <odeds@mellanox.com>, freebsd-infiniband@freebsd.org
Subject:   RE: Infiniband under FreeBSD 11.0-CURRENT...
Message-ID:  <922BDD3D8725118FA5C28939@[10.12.30.106]>
In-Reply-To: <640D4900BB006C2D80760288@[10.12.30.106]>
References:  <F41C4BFDA1646A9029886D1E@Karls-Mac-mini.local> <54AB853C.7070300@selasky.org> <9406DDD429A698CCFE8EA77B@Karls-Mac-mini.local> <54ACE6B1.1060705@selasky.org> <62BC1791A40C62A1836B5368@[10.12.30.106]> <DB3PR05MB026A512731DB21003C9737FDC460@DB3PR05MB026.eurprd05.prod.outlook.com> <640D4900BB006C2D80760288@[10.12.30.106]>
next in thread | previous in thread | raw e-mail | index | archive | help 



--On 07 January 2015 13:45 +0000 Karl Pielorz <kpielorz_lst@tdx.co.uk> 
wrote:

> Ok, I'll set them to ib - and check the relevant modules are loaded,

Ok - sysctl'ing the ports to 'ib' - and opensm / ibstat now see them.

>> Also, please verify the relevant modules are loaded:
>> Kldstat -v | grep 'mlx4\|mlx4ib\|ibcore\|ipoib'

This gives:

    421 ipoib
    420 ibcore
    423 mlx4
    422 mlx4ib

Having configured the interfaces, and run the obligatory 'iperf' on them - 
the results aren't outstanding:

"
[  4] local 10.90.90.2 port 5001 connected with 10.90.90.1 port 64249
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-12.2 sec  1.12 MBytes   771 Kbits/sec

...

[  3] local 10.90.90.1 port 64249 connected with 10.90.90.2 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.7 sec  1.12 MBytes   879 Kbits/sec
"

Under 10.x I know it was better to disable connected mode - is there a way 
this can be done under 11.x - or should I see better performance than that 
using CM?

'ibstat' on the system returns:

"
CA 'mlx4_0'
        CA type: MT25418
        Number of ports: 2
        Firmware version: 2.6.0
        Hardware version: a0
        Node GUID: 0x001a4bffff0c328c
        System image GUID: 0x001a4bffff0c328f
        Port 1:
                State: Active
                Physical state: LinkUp
                Rate: 20
                Base lid: 1
                LMC: 0
                SM lid: 1
                Capability mask: 0x0251086a
                Port GUID: 0x001a4bffff0c328d
        Port 2:
                State: Initializing
                Physical state: LinkUp
                Rate: 20
                Base lid: 0
                LMC: 0
                SM lid: 0
                Capability mask: 0x02510868
                Port GUID: 0x001a4bffff0c328e
"

(Second port is 'Initialising' as I only have one instance of opensm 
running bound to the first port).

Thanks,

-Karl

Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?922BDD3D8725118FA5C28939>