From owner-freebsd-infiniband@FreeBSD.ORG Wed Nov 14 23:32:37 2012 Return-Path: Delivered-To: freebsd-infiniband@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 85272540 for ; Wed, 14 Nov 2012 23:32:37 +0000 (UTC) (envelope-from lists@jnielsen.net) Received: from ns1.jnielsen.net (secure.freebsdsolutions.net [69.55.234.48]) by mx1.freebsd.org (Postfix) with ESMTP id 5F1BB8FC14 for ; Wed, 14 Nov 2012 23:32:36 +0000 (UTC) Received: from [10.10.1.32] (office.betterlinux.com [199.58.199.60]) (authenticated bits=0) by ns1.jnielsen.net (8.14.4/8.14.4) with ESMTP id qAENWN24065819 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Wed, 14 Nov 2012 18:32:23 -0500 (EST) (envelope-from lists@jnielsen.net) Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Subject: Re: using ConnectX card as Ethernet (mlxen) From: John Nielsen In-Reply-To: Date: Wed, 14 Nov 2012 16:32:19 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <53A5EDFD-B2C5-4E24-9AB9-5504AA693602@jnielsen.net> References: <3A359B33-380C-4230-A62C-623765E9376A@jnielsen.net> To: Garrett Cooper X-Mailer: Apple Mail (2.1499) X-DCC-sonic.net-Metrics: ns1.jnielsen.net 1117; Body=2 Fuz1=2 Fuz2=2 X-Virus-Scanned: clamav-milter 0.97.5 at ns1.jnielsen.net X-Virus-Status: Clean Cc: freebsd-infiniband@freebsd.org X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Nov 2012 23:32:37 -0000 On Nov 9, 2012, at 9:51 PM, Garrett Cooper wrote: > On Fri, Nov 9, 2012 at 7:44 PM, John Nielsen = wrote: > On the off chance there's someone on this list already that didn't see = my earlier (unanswered) question, I'm reposting it below. I don't have = FreeBSD on this hardware any more but I would like to know for future = reference if using these cards with an Ethernet switch is an option. >=20 > I know this might seem dumb, but what if you did = sys.device.mlx4_core0.mlx4_port1=3D"`printf 'en\n'`" (the newline's = important). I'll try it next time I have an opportunity. Can you explain why the = newline might make a difference? Also, I don't think I ever tried 'en', = just 'eth'. Can someone say what the right value is or should be? Thanks, JN From owner-freebsd-infiniband@FreeBSD.ORG Wed Nov 14 23:53:15 2012 Return-Path: Delivered-To: freebsd-infiniband@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 14DCAAD4 for ; Wed, 14 Nov 2012 23:53:15 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id C0E0C8FC13 for ; Wed, 14 Nov 2012 23:53:14 +0000 (UTC) Received: by mail-ob0-f182.google.com with SMTP id 16so1377920obc.13 for ; Wed, 14 Nov 2012 15:53:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Dc2Khwt1NVFbpo6Uj1bZSld3zfprU6Nhme/q9JaSsA8=; b=UFCCXIZnNkVdt5I28/JYXurOzmFrJAgk3C8f3aGYtMo4YIJThxuaae/CldI93l7szt NKq7kRpqrY0eoZ64SQVh7NErrjMzQsp+Wn/8ku3YJgE1H9m2taa304pPvO1O0k7xwriX J58m39S+VoneuxGPgIlMRfeTmiJm4ew6osoTL82/fhNnzrlD1Ei8tjRpvvRkckJgwv5M /QYpan2IFKXj77wgWizkLsk+ixObwqJ1dKSXF+tBrULjdkATw3c0p1Wyfnt5277o7gLO SbDlt15zBP4k7LuzatSb3D+dhkQw7uwqHZHZxaPPDFz2umO54BFQR6fmaQpGNOHE4a3u xSMA== MIME-Version: 1.0 Received: by 10.60.32.137 with SMTP id j9mr22405188oei.133.1352937194074; Wed, 14 Nov 2012 15:53:14 -0800 (PST) Received: by 10.76.143.33 with HTTP; Wed, 14 Nov 2012 15:53:13 -0800 (PST) In-Reply-To: <53A5EDFD-B2C5-4E24-9AB9-5504AA693602@jnielsen.net> References: <3A359B33-380C-4230-A62C-623765E9376A@jnielsen.net> <53A5EDFD-B2C5-4E24-9AB9-5504AA693602@jnielsen.net> Date: Wed, 14 Nov 2012 15:53:13 -0800 Message-ID: Subject: Re: using ConnectX card as Ethernet (mlxen) From: Garrett Cooper To: John Nielsen Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-infiniband@freebsd.org X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Nov 2012 23:53:15 -0000 On Wed, Nov 14, 2012 at 3:32 PM, John Nielsen wrote: > On Nov 9, 2012, at 9:51 PM, Garrett Cooper wrote: > > > On Fri, Nov 9, 2012 at 7:44 PM, John Nielsen wrote: > > On the off chance there's someone on this list already that didn't see > my earlier (unanswered) question, I'm reposting it below. I don't have > FreeBSD on this hardware any more but I would like to know for future > reference if using these cards with an Ethernet switch is an option. > > > > I know this might seem dumb, but what if you did > sys.device.mlx4_core0.mlx4_port1="`printf 'en\n'`" (the newline's > important). > > I'll try it next time I have an opportunity. Can you explain why the > newline might make a difference? Also, I don't think I ever tried 'en', > just 'eth'. Can someone say what the right value is or should be? > It was a WAG based on this section of code (from sys/ofed/drivers/net/mlx4/main.c)... 469 static ssize_t set_port_type(struct device *dev, 470 struct device_attribute *attr, 471 const char *buf, size_t count) 472 { 473 struct mlx4_port_info *info = container_of(attr, struct mlx4_port_i nfo, 474 port_attr); 475 struct mlx4_dev *mdev = info->dev; 476 struct mlx4_priv *priv = mlx4_priv(mdev); 477 enum mlx4_port_type types[MLX4_MAX_PORTS]; 478 enum mlx4_port_type new_types[MLX4_MAX_PORTS]; 479 int i; 480 int err = 0; 481 482 if (!strcmp(buf, "ib\n")) 483 info->tmp_type = MLX4_PORT_TYPE_IB; 484 else if (!strcmp(buf, "eth\n")) 485 info->tmp_type = MLX4_PORT_TYPE_ETH; 486 else if (!strcmp(buf, "auto\n")) 487 info->tmp_type = MLX4_PORT_TYPE_AUTO; 488 else { 489 mlx4_err(mdev, "%s is not supported port type\n", buf); 490 return -EINVAL; 491 } I don't have one of these cards so I can't verify whether or not that's the issue today, but it seemed like a likely cause. This being there is probably the remnant of a pseudo-valid hacky assumption on Linux (you're echo'ing to a sysfs node and it's tacking on a newline), but an invalid assumption on FreeBSD (the sysctl<->sysfs compat layer doesn't make any translation assumptions w.r.t. newlines, etc). Thanks! -Garrett