From owner-freebsd-infiniband@FreeBSD.ORG Tue May 28 15:56:52 2013 Return-Path: Delivered-To: freebsd-infiniband@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 6CF4990E; Tue, 28 May 2013 15:56:52 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-bk0-x22b.google.com (mail-bk0-x22b.google.com [IPv6:2a00:1450:4008:c01::22b]) by mx1.freebsd.org (Postfix) with ESMTP id 9D6CF3F3; Tue, 28 May 2013 15:56:51 +0000 (UTC) Received: by mail-bk0-f43.google.com with SMTP id jm2so2730422bkc.30 for ; Tue, 28 May 2013 08:56:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=uL2xTPn1L5HiYo/HocjOlHx4E7saC7x/Rg5jgaQEpI4=; b=VFJobJWgLI8bu0X7yxVPjS7ki9SelCq9R96CS/TsywqV+WnVBSUzr2kPmSvRmuK4cA pOwbO3dn4e9Slie7HAyWjRUUTflSzmeSS0XwrEEK9dvFsrHqnKuPl/wNsiBHcRLgo+J9 G+lVabbCy6KY5DACqHHFCsRuSLMrl6TlWzBWBzjVGzhQckisIjLQBVpq++FLdN+vUpk4 STQSYto+rzotBV3eb+G773XWDdU/hoEeoar6ZTFPpOX1FdzOMddJWwB6kqgs/YXwfmeR hAJhjRPVo5Yp0PxZkoDS/L36z3nIWjSaj3gYRWQuyH/wIwuChjs5agSNwM8rJgEPd23d 33Og== MIME-Version: 1.0 X-Received: by 10.204.235.129 with SMTP id kg1mr13508098bkb.28.1369756610195; Tue, 28 May 2013 08:56:50 -0700 (PDT) Received: by 10.205.40.129 with HTTP; Tue, 28 May 2013 08:56:50 -0700 (PDT) In-Reply-To: <64DAB3164E410447932305F50F896D8D6AF65A33@MTLDAG01.mtl.com> References: <64DAB3164E410447932305F50F896D8D6AF65A33@MTLDAG01.mtl.com> Date: Tue, 28 May 2013 08:56:50 -0700 Message-ID: Subject: Re: Create bond on Infiniband ports From: Garrett Cooper To: Alex Liptsin Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-infiniband@freebsd.org" , Regev Lev , "freebsd-questions@freebsd.org" , "freebsd-net@freebsd.org" X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 May 2013 15:56:52 -0000 On Tue, May 28, 2013 at 4:35 AM, Alex Liptsin wrote: > Hi. > > I use FreeBSD 9.1 with OFED compiled on it. > > There is a Mellanox adapter: > > [root@qa-h-vrt-030-006 ~]# pciconf -lv |grep mlx4 -A 3 > mlx4_core0@pci0:0:5:0: class=0x028000 card=0x005015b3 chip=0x100315b3 > rev=0x00 hdr=0x00 > vendor = 'Mellanox Technologies' > device = 'MT27500 Family [ConnectX-3]' > class = network > > I want to create Bond on the two ports (ib0 and ib1) of this device: > > [root@qa-h-vrt-030-006 ~]# ifconfig > em0: flags=8843 metric 0 mtu 1500 > > options=209b > ether 00:50:56:23:1e:06 > inet6 fe80::250:56ff:fe23:1e06%em0 prefixlen 64 scopeid 0x2 > inet 10.195.30.6 netmask 0xffff0000 broadcast 10.195.255.255 > nd6 options=23 > media: Ethernet autoselect (1000baseT ) > status: active > lo0: flags=8049 metric 0 mtu 16384 > options=600003 > inet6 ::1 prefixlen 128 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 > inet 127.0.0.1 netmask 0xff000000 > nd6 options=23 > ib0: flags=8043 metric 0 mtu 65520 > options=80018 > lladdr 80.0.0.48.fe.80.0.0.0.0.0.0.0.2.c9.0.1.0.d0.51 > inet 11.195.30.1 netmask 0xffff0000 broadcast 11.195.255.255 > inet6 fe80::250:56ff:fe23:1e06%ib0 prefixlen 64 scopeid 0x4 > nd6 options=23 > ib1: flags=8043 metric 0 mtu 65520 > options=80018 > lladdr 80.0.0.49.fe.80.0.0.0.0.0.0.0.2.c9.0.1.0.d0.52 > inet 12.195.30.1 netmask 0xffff0000 broadcast 12.195.255.255 > inet6 fe80::250:56ff:fe23:1e06%ib1 prefixlen 64 scopeid 0x5 > nd6 options=23 > > > 1. Is there "bond" in FreeBSD or its "Lagg"? > It's lagg. Setting up bonded interfaces is different (Anthony C. or someone else can provide more details about this). failover mode via lagg is completely different from failover mode with IB IIRC. > 2. I tried to create this way, but failed: > > [root@qa-h-vrt-030-006 ~]# ifconfig lagg0 create > > root@qa-h-vrt-031-005 conf]# ifconfig lagg0 laggproto failover laggport > ib0 laggport ib1 > > ifconfig: SIOCSLAGGPORT: Protocol not supported > > Any ideas? > Is it supported on Infiniband ports? > The media setting pieces are look to only be supported with mlx4 in en mode: 1076 static int mlx4_en_ioctl(struct ifnet *dev, u_long command, caddr_t data) 1077 { ... 1115 case SIOCGIFMEDIA: 1116 error = ifmedia_ioctl(dev, ifr, &priv->media, command); 1117 break; If I remember correctly, IB ports with mlx4 default to ib mode, not en mode (it also helps to have the right drivers loaded for this). Cheers, -Garrett