From owner-freebsd-net@FreeBSD.ORG Sun May 26 11:44:52 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 58D18FE7; Sun, 26 May 2013 11:44:52 +0000 (UTC) (envelope-from alexl@mellanox.com) Received: from eu1sys200aog103.obsmtp.com (eu1sys200aog103.obsmtp.com [207.126.144.115]) by mx1.freebsd.org (Postfix) with ESMTP id 116F0315; Sun, 26 May 2013 11:44:50 +0000 (UTC) Received: from MTLCAS02.mtl.com ([193.47.165.155]) (using TLSv1) by eu1sys200aob103.postini.com ([207.126.147.11]) with SMTP ID DSNKUaH1rsaeVRw+D1E18SnbUWBUvFyJblXa@postini.com; Sun, 26 May 2013 11:44:51 UTC Received: from MTLDAG01.mtl.com ([10.0.8.75]) by MTLCAS02.mtl.com ([10.0.8.72]) with mapi id 14.03.0123.003; Sun, 26 May 2013 14:43:30 +0300 From: Alex Liptsin To: "freebsd-questions@freebsd.org" , "freebsd-net@freebsd.org" Subject: How to switch Datgram/Connected mtu modes? Thread-Topic: How to switch Datgram/Connected mtu modes? Thread-Index: Ac5aBjoH5e0b3u1XTvyAlDGQDXzaKQ== Date: Sun, 26 May 2013 11:43:29 +0000 Message-ID: <64DAB3164E410447932305F50F896D8D6AF651AD@MTLDAG01.mtl.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.0.13.1] MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 May 2013 11:44:52 -0000 Hello. I work with FreeBSD 9.1 and Mellanox devices. How can I configure MTU in connected mode on FreeBSD 9.1? In Linux to enable connected mode for interface ib0, I enter: echo connected > /sys/class/net/ib0/mode Switching between CM and UD mode can be done in run time: echo datagram > /sys/class/net/ib0/mode sets the mode of ib0 to UD echo connected > /sys/class/net/ib0/mode sets the mode ib0 to CM There is no such directories at FreeBSD. Wat shall I do? Datagram vs Connected modes The IPoIB driver supports two modes of operation: datagram and connected. The mode is set and read through an interface's /sys/class/net//mode file. In datagram mode, the IB UD (Unreliable Datagram) transport is used and so the interface MTU has is equal to the IB L2 MTU minus the IPoIB encapsulation header (4 bytes). For example, in a typical IB fabric with a 2K MTU, the IPoIB MTU will be 2048 - 4 =3D 2044 bytes. In connected mode, the IB RC (Reliable Connected) transport is used. Connected mode takes advantage of the connected nature of the IB transport and allows an MTU up to the maximal IP packet size of 64K, which reduces the number of IP packets needed for handling large UDP datagrams, TCP segments, etc and increases the performance for large messages. In connected mode, the interface's UD QP is still used for multicast and communication with peers that don't support connected mode. In this case, RX emulation of ICMP PMTU packets is used to cause the networking stack to use the smaller UD MTU for these neighbours. Thanks a lot Regards, Alex Liptsin Office: +972 (74) 7236141 Mobile: +972(54) 7833986 Fax: +972(74) 7236161 Email: alexl@mellanox.com