Date: Tue, 28 May 2013 08:56:50 -0700 From: Garrett Cooper <yanegomi@gmail.com> To: Alex Liptsin <alexl@mellanox.com> Cc: "freebsd-infiniband@freebsd.org" <freebsd-infiniband@freebsd.org>, Regev Lev <regevl@mellanox.com>, "freebsd-questions@freebsd.org" <freebsd-questions@freebsd.org>, "freebsd-net@freebsd.org" <freebsd-net@freebsd.org> Subject: Re: Create bond on Infiniband ports Message-ID: <CAGH67wSJFhtrc=K-QVAT3va6Uz7r-8X%2B06G0vh15GAu%2BXToWug@mail.gmail.com> In-Reply-To: <64DAB3164E410447932305F50F896D8D6AF65A33@MTLDAG01.mtl.com> References: <64DAB3164E410447932305F50F896D8D6AF65A33@MTLDAG01.mtl.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, May 28, 2013 at 4:35 AM, Alex Liptsin <alexl@mellanox.com> wrote:
> Hi.
>
> I use FreeBSD 9.1 with OFED compiled on it.
>
> There is a Mellanox adapter:
>
> [root@qa-h-vrt-030-006 ~]# pciconf -lv |grep mlx4 -A 3
> mlx4_core0@pci0:0:5:0: class=0x028000 card=0x005015b3 chip=0x100315b3
> rev=0x00 hdr=0x00
> vendor = 'Mellanox Technologies'
> device = 'MT27500 Family [ConnectX-3]'
> class = network
>
> I want to create Bond on the two ports (ib0 and ib1) of this device:
>
> [root@qa-h-vrt-030-006 ~]# ifconfig
> em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
>
> options=209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC>
> ether 00:50:56:23:1e:06
> inet6 fe80::250:56ff:fe23:1e06%em0 prefixlen 64 scopeid 0x2
> inet 10.195.30.6 netmask 0xffff0000 broadcast 10.195.255.255
> nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
> media: Ethernet autoselect (1000baseT <full-duplex>)
> status: active
> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
> options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
> inet6 ::1 prefixlen 128
> inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
> inet 127.0.0.1 netmask 0xff000000
> nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
> ib0: flags=8043<UP,BROADCAST,RUNNING,MULTICAST> metric 0 mtu 65520
> options=80018<VLAN_MTU,VLAN_HWTAGGING,LINKSTATE>
> lladdr 80.0.0.48.fe.80.0.0.0.0.0.0.0.2.c9.0.1.0.d0.51
> inet 11.195.30.1 netmask 0xffff0000 broadcast 11.195.255.255
> inet6 fe80::250:56ff:fe23:1e06%ib0 prefixlen 64 scopeid 0x4
> nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
> ib1: flags=8043<UP,BROADCAST,RUNNING,MULTICAST> metric 0 mtu 65520
> options=80018<VLAN_MTU,VLAN_HWTAGGING,LINKSTATE>
> lladdr 80.0.0.49.fe.80.0.0.0.0.0.0.0.2.c9.0.1.0.d0.52
> inet 12.195.30.1 netmask 0xffff0000 broadcast 12.195.255.255
> inet6 fe80::250:56ff:fe23:1e06%ib1 prefixlen 64 scopeid 0x5
> nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
>
>
> 1. Is there "bond" in FreeBSD or its "Lagg"?
>
It's lagg. Setting up bonded interfaces is different (Anthony C. or someone
else can provide more details about this). failover mode via lagg is
completely different from failover mode with IB IIRC.
> 2. I tried to create this way, but failed:
>
> [root@qa-h-vrt-030-006 ~]# ifconfig lagg0 create
>
> root@qa-h-vrt-031-005 conf]# ifconfig lagg0 laggproto failover laggport
> ib0 laggport ib1
>
> ifconfig: SIOCSLAGGPORT: Protocol not supported
>
> Any ideas?
> Is it supported on Infiniband ports?
>
The media setting pieces are look to only be supported with mlx4 in en mode:
1076 static int mlx4_en_ioctl(struct ifnet *dev, u_long command, caddr_t
data)
1077 {
...
1115 case SIOCGIFMEDIA:
1116 error = ifmedia_ioctl(dev, ifr, &priv->media, command);
1117 break;
If I remember correctly, IB ports with mlx4 default to ib mode, not en mode
(it also helps to have the right drivers loaded for this).
Cheers,
-Garrett
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAGH67wSJFhtrc=K-QVAT3va6Uz7r-8X%2B06G0vh15GAu%2BXToWug>