From owner-freebsd-infiniband@FreeBSD.ORG Wed Feb 4 23:26:18 2015 Return-Path: Delivered-To: freebsd-infiniband@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6DBD93BA for ; Wed, 4 Feb 2015 23:26:18 +0000 (UTC) Received: from na01-bl2-obe.outbound.protection.outlook.com (mail-bl2on0081.outbound.protection.outlook.com [65.55.169.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E74358FF for ; Wed, 4 Feb 2015 23:26:17 +0000 (UTC) Received: from DM2PR0801MB1037.namprd08.prod.outlook.com (25.160.133.25) by DM2PR0801MB1086.namprd08.prod.outlook.com (25.160.134.144) with Microsoft SMTP Server (TLS) id 15.1.75.20; Wed, 4 Feb 2015 23:26:08 +0000 Received: from DM2PR0801MB1039.namprd08.prod.outlook.com (25.160.133.27) by DM2PR0801MB1037.namprd08.prod.outlook.com (25.160.133.25) with Microsoft SMTP Server (TLS) id 15.1.81.19; Wed, 4 Feb 2015 23:26:06 +0000 Received: from DM2PR0801MB1039.namprd08.prod.outlook.com ([25.160.133.27]) by DM2PR0801MB1039.namprd08.prod.outlook.com ([25.160.133.27]) with mapi id 15.01.0081.018; Wed, 4 Feb 2015 23:26:06 +0000 From: "Lewis, Fred" To: "freebsd-infiniband@freebsd.org" Subject: Does mlxen work on FreeBSD 10.1 with ConnectX-3 Thread-Topic: Does mlxen work on FreeBSD 10.1 with ConnectX-3 Thread-Index: AQHQQNHyTj4Ou0Fn6kKQiYrjjngl2Q== Date: Wed, 4 Feb 2015 23:26:06 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [64.80.217.3] authentication-results: freebsd.org; dkim=none (message not signed) header.d=none; x-microsoft-antispam: BCL:0;PCL:0;RULEID:;SRVR:DM2PR0801MB1037;UriScan:; x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:; SRVR:DM2PR0801MB1037; x-forefront-prvs: 04772EA191 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6009001)(164054003)(36756003)(229853001)(19580395003)(19580405001)(106116001)(122556002)(110136001)(107886001)(2351001)(2900100001)(86362001)(77156002)(92566002)(450100001)(16236675004)(46102003)(62966003)(102836002)(87936001)(50986999)(54356999)(40100003)(66066001)(2656002)(94096001); DIR:OUT; SFP:1101; SCL:1; SRVR:DM2PR0801MB1037; H:DM2PR0801MB1039.namprd08.prod.outlook.com; FPR:; SPF:None; MLV:sfv; LANG:en; MIME-Version: 1.0 X-MS-Exchange-CrossTenant-originalarrivaltime: 04 Feb 2015 23:26:06.1091 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: acf01c9d-c699-42af-bdbb-44bf582e60b0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM2PR0801MB1037 X-Microsoft-Antispam: BCL:0;PCL:0;RULEID:;SRVR:DM2PR0801MB1086; X-OriginatorOrg: panasas.com Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Feb 2015 23:26:18 -0000 Hi FreeBSD IB folks, I'm trying to bring up a Mellanox Ethernert Interface and it is showing as = "no carrier." The cables were swapped out with a known good one and it didn't seem to mak= e any difference. The remote peer port is admin UP. I've built the KLDs using the latest FreeBSD OFED stuff (v2.1.5). The build machine is different from the machine that I'm loading the KLDs o= n. The build machine does not have any OFED options or devices installed. [mlxen]$ sysctl kern.conftxt | grep -i ofed [mlxen]$ I built the KLDs as follows: cd mlx4 ; make clean cleandepend; make cd mlxen; make clean cleandepend; make I then copied the KLDs (mlx4.ko and mlxen.ko) to /boot/kernel on the target= machine and did a kldload. The modules loaded fine as far as I can tell and mlxen0 = shows up in the interface list. mlx4_core0@pci0:5:0:0: class=3D0x028000 card=3D0x100315b3 chip=3D0x100315b= 3 rev=3D0x00 hdr=3D0x00 vendor =3D 'Mellanox Technologies' device =3D 'MT27500 Family [ConnectX-3]' class =3D network root@mau-da-27-4-3:/boot/kernel # kldload mlx4.ko; kldload mlxen.ko mlx4_core0: mem 0x93a00000-0x93afffff,0x383fff000000-0x383fff7f= ffff irq 42 at device 0.0 on pci5 <6>mlx4_core: Mellanox ConnectX core driver v2.1 (Feb 4 2015) <6>mlx4_en: Mellanox ConnectX HCA Ethernet driver v2.1 (Feb 4 2015) mlxen0: Ethernet address: 00:1e:67:81:40:61 mau-da-27-4-3 kemlrnel: mlx4_en: mx4lx4_core0: Port _e1:= Using 24 TX rn:ings mlx4_core0: Port 1: Using 24 TX rings <4>mlx4_en: mlx4_core0: Port 1: Using 16 RX rings mau-da-27-4-3 kemlrnel: mlx4_en: mx4lx4_core0: Port _e1:= Using 16 RX rn:ings Ub 4 14 m:23:18 mau-da-27lx-4-3 kernel: mlxen4_en: mlxen0: Us0:ing 24 TX = rings sing 24 TX rings <4>mlx4_en: mlxen0: Using 16 RX rings Feb 4 14:23:18 <4mau-da-27-4-3 ke>mrnel: mlx4_en: mlxlxen0: Using 16 4_RX = rings en: mlxen0: Initializing port Feb 4 14:23:19 mau-da-27-4-3 kernel: mlx4_en: mlxen0: Initializing port root@mau-da-27-4-3:/boot/kernel # kldstat -v | grep 'mlx4\|mlxen' 9 2 0xffffffff81c6c000 37816 mlx4.ko (/boot/kernel/mlx4.ko) 494 mlx4 10 1 0xffffffff81ca4000 125e2 mlxen.ko (/boot/kernel/mlxen.ko) 495 mlxen ifconfig -a mlxen0: flags=3D8843 metric 0 mtu 1= 500 options=3Dd07bb ether 00:1e:67:81:40:61 nd6 options=3D29 media: Ethernet autoselect status: no carrier root@mau-da-27-4-3:/boot/kernel # sysctl -a | grep mlx4 hw.mlxen0.conf.device_name: mlx4_core0 dev.mlx4_core.%parent: dev.mlx4_core.0.%desc: mlx4_core dev.mlx4_core.0.%driver: mlx4_core dev.mlx4_core.0.%location: slot=3D0 function=3D0 handle=3D\_SB_.PCI0.BR3C.H= 000 dev.mlx4_core.0.%pnpinfo: vendor=3D0x15b3 device=3D0x1003 subvendor=3D0x15b= 3 subdevice=3D0x1003 class=3D0x028000 dev.mlx4_core.0.%parent: pci5 sys.device.mlx4_core0.rate_limit_caps.different_rates_num: 0 sys.device.mlx4_core0.mlx4_port1: eth If anyone can see anything I am missing or doing wrong, I'd be glad to know= . Thanks, -Fred