Date: Thu, 6 Dec 2012 19:49:21 +0900 From: Daichi GOTO <daichi@bsd-consulting.jp> To: infiniband@freebsd.org Cc: Masanori OZAWA <ozawa@ongs.net>, hrs@allbsd.org Subject: IB test report : 20121206 Message-ID: <20121206194921.12f07e3d889c9d8cf5a44967@bsd-consulting.jp>
next in thread | raw e-mail | index | archive | help
Hi guys, ================================================================ IB test report. OS and IB cards: FreeBSD9-stable r243935 MHQH19B-XTR (ConnectX2) MCX353A-QCBT (ConnectX2) ================================================================ TEST1 IBoIP UD (Unreliable Datagram) Kernel configuration include GENERIC ident INFINIBAND options OFED # Infiniband protocol options SDP # Sockets Direct Protocol for infiniband device mlx4ib # ConnectX Infiniband support device mlxen # ConnectX Ethernet support device mthca # Infinihost cards device ipoib # IP over IB devices make.conf WITH_OFED= YES rc.conf (server side) opensm_enable="YES" IP setting ifconfig ib0 mtu 2044 ifconfig ib0 inet 192.168.2.x netmask 0xffffff00 server: iperf bench # iperf -s ------------------------------------------------------------ Server listening on TCP port 5001 TCP window size: 64.0 KByte (default) ------------------------------------------------------------ [ 4] local 192.168.2.1 port 5001 connected with 192.168.2.2 port 12551 [ ID] Interval Transfer Bandwidth [ 4] 0.0-10.0 sec 6.00 GBytes 5.15 Gbits/sec client: iperf bench # iperf -c 192.168.2.1 -i 1 ------------------------------------------------------------ Client connecting to 192.168.2.1, TCP port 5001 TCP window size: 33.1 KByte (default) ------------------------------------------------------------ [ 3] local 192.168.2.2 port 12551 connected with 192.168.2.1 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0- 1.0 sec 47.2 MBytes 396 Mbits/sec [ 3] 1.0- 2.0 sec 49.0 MBytes 411 Mbits/sec [ 3] 2.0- 3.0 sec 260 MBytes 2.18 Gbits/sec [ 3] 3.0- 4.0 sec 880 MBytes 7.39 Gbits/sec [ 3] 4.0- 5.0 sec 926 MBytes 7.77 Gbits/sec [ 3] 5.0- 6.0 sec 879 MBytes 7.38 Gbits/sec [ 3] 6.0- 7.0 sec 464 MBytes 3.90 Gbits/sec [ 3] 7.0- 8.0 sec 947 MBytes 7.95 Gbits/sec [ 3] 8.0- 9.0 sec 945 MBytes 7.93 Gbits/sec [ 3] 9.0-10.0 sec 742 MBytes 6.22 Gbits/sec [ 3] 0.0-10.0 sec 6.00 GBytes 5.15 Gbits/sec result: stable transmission. average 800MB/sec-1000MB/sec mtu 2044 is max. ================================================================ TEST2 IBoIP CM Kernel configuration include GENERIC ident INFINIBAND options OFED # Infiniband protocol options SDP # Sockets Direct Protocol for infiniband options IPOIB_CM#Use connect mode ipoib device mlx4ib # ConnectX Infiniband support device mlxen # ConnectX Ethernet support device mthca # Infinihost cards device ipoib # IP over IB devices others the same result unstable. terrible. average 0MB/sec-1000MB/sec ================================================================ Consideration from this first experimetns For OFED1.5.x of FreeBSD and ConnectX2 cards, Unreliable Datagram mode IB over IP looks the only way to use. CM is very unstable. It's odd. Needs improvement. (We must improve around mtu code? 2044 is dinky) I'm wondering that how many developers are using ConnectX2 cards with CM OFED kernel. ================================================================ -- Daichi GOTO Director | BSD Consulting Inc. 3F SS bldg. 3-4-2, Nishishinbashi, Minato, Tokyo 81-80-4174-6161 | daichi@bsd-consulting.jp
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20121206194921.12f07e3d889c9d8cf5a44967>