From owner-freebsd-infiniband@FreeBSD.ORG Wed Dec 12 22:06:07 2012 Return-Path: Delivered-To: freebsd-infiniband@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id AE02EB1C for ; Wed, 12 Dec 2012 22:06:07 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-lb0-f182.google.com (mail-lb0-f182.google.com [209.85.217.182]) by mx1.freebsd.org (Postfix) with ESMTP id 2165D8FC0A for ; Wed, 12 Dec 2012 22:06:06 +0000 (UTC) Received: by mail-lb0-f182.google.com with SMTP id go10so1170425lbb.13 for ; Wed, 12 Dec 2012 14:06:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=0lNzIplDajaTtGMgMFvKG74gqMzZQBI//4mgsaGRE70=; b=WPv/xu4mmhnwAxiR9XhJpC8nrrf7bYxRPqKX51MvfKQjQEsnsO1fw1VB4LhgtfE4M5 0GS6DsrEXYOhCXu0YHXjohP/ZN5J6lDJLdNmaohTRLU9EzH68SigAC9JLAEQpdZaJdlJ //ZJu8+qAwjp+EpMo2wP/sv0Ne61rLKMAv9DwGdNjbGvPqf6CPr5KcGhTSJBPtrQotaz VNrL1+LgPqUXjpcswENStbKTvk/WmxWIN8jxDiMedtbAalctQXOiDPv74jmZKa1emfmy nBCnVr0OTrHvR6Je1o95DS6t8Uo63Ldbng3djY74h98eUWtb99cTaMlDWvTEdZPNo8iP EDDA== MIME-Version: 1.0 Received: by 10.152.105.103 with SMTP id gl7mr2519528lab.10.1355349965687; Wed, 12 Dec 2012 14:06:05 -0800 (PST) Received: by 10.112.99.70 with HTTP; Wed, 12 Dec 2012 14:06:05 -0800 (PST) In-Reply-To: References: <3A359B33-380C-4230-A62C-623765E9376A@jnielsen.net> <53A5EDFD-B2C5-4E24-9AB9-5504AA693602@jnielsen.net> Date: Wed, 12 Dec 2012 14:06:05 -0800 Message-ID: Subject: Re: using ConnectX card as Ethernet (mlxen) From: Garrett Cooper To: Anthony Cornehl Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-infiniband@freebsd.org X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Dec 2012 22:06:07 -0000 On Thu, Dec 6, 2012 at 8:07 PM, Anthony Cornehl wrote: > There is an issue with the sysfs/sysctl abstraction for storing values. > Whups. > > http://www.freebsd.org/cgi/query-pr.cgi?pr=174213 I have a patch for the sysctl<->sysfs shim that corrects it to function when storing/showing values and makes it function more like Linux. Setting the value works on CURRENT when changing to eth mode a single caveat: when changing the mode for the card, you need to do it for the last port. This is because of the following code [in sys/ofed/drivers/net/mlx4/main.c]: 200 int mlx4_check_port_params(struct mlx4_dev *dev, 201 enum mlx4_port_type *port_type) 202 { 203 int i; ... 212 if (port_type[i] == MLX4_PORT_TYPE_ETH && 213 port_type[i + 1] == MLX4_PORT_TYPE_IB) 214 return -EINVAL; /* <-- HERE */ 215 } 216 } Example (setting the first port to eth mode when the other is in ib mode): # sysctl sys.device.mlx4_core0 sys.device.mlx4_core0.mlx4_port1: ib sys.device.mlx4_core0.mlx4_port2: ib # sysctl sys.device.mlx4_core0.mlx4_port1=eth sys.device.mlx4_core0.mlx4_port1: ib sysctl: sys.device.mlx4_core0.mlx4_port1: Invalid argument Example (setting the last port to eth mode when the first one is in ib mode flips the mode of the entire card to eth mode): # sysctl sys.device.mlx4_core0.mlx4_port2=eth sys.device.mlx4_core0.mlx4_port2: ib -> eth # sysctl sys.device.mlx4_core0 sys.device.mlx4_core0.mlx4_port1: eth sys.device.mlx4_core0.mlx4_port2: eth Need to functionally test whether or not it's functioning properly with eth mode, but at least you can set it with my upcoming patch. I'm sending the code out to our IB team and submitting the patch to jeff@ for review. Thanks, -Garrett