Date: Mon, 5 Dec 2016 17:08:00 +0000 From: =?utf-8?B?Vmluw61jaXVzIEZlcnLDo28=?= <ferrao@vmio.com.br> To: "freebsd-infiniband@freebsd.org" <freebsd-infiniband@freebsd.org> Subject: How to change between Connected Mode and Datagram Mode with IPoIB? Message-ID: <0EC3E7A7-497B-4961-BB14-DA5B549E6750@vmio.com.br>
next in thread | raw e-mail | index | archive | help
Hello guys, Continuing my journey on Infiniband on FreeBSD I’ve discovered that some bugs and performance issues appears to be a mismatch on CM and UD modes of IPoIB. I was able to raise the performance of NFS to 750MB/s changing the Linux connection mode to CM. After that the MTU changed from 2044 to 65520 and I saw some good values. But another annoyances appeared on Linux side. A sluggish performance when running simple commands like “ls” on NFS over IB mounts rendered the solution inviable. So I’m considering running FreeBSD in UD (Datagram) mode to see if I finally can get rid of the problems. But, how can I change between CM and UD in FreeBSD? We don't have a /sys folder on FreeBSD so I cant just set from 0 to 1 and change modes. Some old forum posts mentions a recompile needed, disabling "options IPOIB_CM” on kernel configuration just to change modes, which is pretty inviable, and it doesn’t seems right. So how can we achieve changing between CM and UD modes? And one last question: how it was even working (badly) but working when a side is talking in CM and the other side on UD mode? Thanks in advance, V.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?0EC3E7A7-497B-4961-BB14-DA5B549E6750>