From owner-freebsd-infiniband@FreeBSD.ORG Sat Nov 10 01:12:56 2012 Return-Path: Delivered-To: freebsd-infiniband@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A6939519; Sat, 10 Nov 2012 01:12:56 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-oa0-f54.google.com (mail-oa0-f54.google.com [209.85.219.54]) by mx1.freebsd.org (Postfix) with ESMTP id 072B88FC08; Sat, 10 Nov 2012 01:12:46 +0000 (UTC) Received: by mail-oa0-f54.google.com with SMTP id n9so5796433oag.13 for ; Fri, 09 Nov 2012 17:12:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:cc:content-type; bh=dfaB4bPnArkab8sdJg/Hd4dR/DKKAvFUZ2ACsQpf2gk=; b=TbuQJtxnWW524lkbw7NIfH/PdU817HZ6aQG7bcyHVDovM2kvd/UxXFCayKkqndZya6 4fLv8Jli2hq+ZB2HkyvE+GidSjQ2bCdMkzlHK1Bu4NhBry4Tz5V6aEi21ZOKrXZ0EBE1 1h31mc4ooRqHJvuBTjCpLY1PKjx2/j2nEV4DmXrttUNaG8pYQaaB7UPWvsmpFSAKaUGE rS/DibIeZGgoTwG/VCNCH7LfCdoAH0H74W/MhqpsnZpij5Fg1Ntp0RcDpB8xITcNippu 2LUDrO0cNjQoLYVxys4+aGmmqyNM31VZHOLw8Y49h6CoAdEt8PhgG4vdPtmm/ZUULhj1 Wmgw== MIME-Version: 1.0 Received: by 10.60.11.105 with SMTP id p9mr9500158oeb.128.1352509965721; Fri, 09 Nov 2012 17:12:45 -0800 (PST) Received: by 10.76.143.33 with HTTP; Fri, 9 Nov 2012 17:12:45 -0800 (PST) Date: Fri, 9 Nov 2012 17:12:45 -0800 Message-ID: Subject: Infiniband list available From: Garrett Cooper To: vendors@FreeBSD.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: network@FreeBSD.org, freebsd-infiniband@FreeBSD.org X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Nov 2012 01:12:56 -0000 Hi all, With the help of postmaster, a new Infiniband list has been created for FreeBSD. If you are interested in IB on FreeBSD, please subscribe to the list. My group (Isilon) will be providing more details about the porting effort being done from our end to the latest 1.5.x and the group is working at improving IB documentation and the like in the interim. Feel free to discuss anything IB related on the list! Cheers! -Garrett From owner-freebsd-infiniband@FreeBSD.ORG Sat Nov 10 01:19:33 2012 Return-Path: Delivered-To: freebsd-infiniband@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B5BB8580; Sat, 10 Nov 2012 01:19:33 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-oa0-f54.google.com (mail-oa0-f54.google.com [209.85.219.54]) by mx1.freebsd.org (Postfix) with ESMTP id 682F68FC13; Sat, 10 Nov 2012 01:19:29 +0000 (UTC) Received: by mail-oa0-f54.google.com with SMTP id n9so5800484oag.13 for ; Fri, 09 Nov 2012 17:19:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:cc:content-type; bh=ca0ac+inXFkzNPj88ED0IDDiyPdZyjgr89A+F2aptvk=; b=hW1bjhxUOQpIN6uimsGZrmCfy7MmEz9Y4jJvVAhIuG9XG7H9lfbBsITVlMFGVlEUWC glx1b6JR3kp02IV7aqYfbYA8+HYTH/5j7R9EftD0o8PIEZFBIBAJk57n0Bss8elXwBpZ 4giIo1vPG5NkhUyvysUjs+dpGQXntWSJ3I/YLQmSsw7J9a/DNBD3HEOdps2EK0RAhA93 TXA06LviudrDo8XxdXVNJOCtkx/R7nqf+IKMFulZn34NBp3xN2fCVAePzfnKNP95ivvQ roMwYmRbOawUCpE4Ez0mS7oaWmFOEv1RNrToiKLzb7AKwDsyvX6fspY7cCePURN1iBec qpFw== MIME-Version: 1.0 Received: by 10.182.95.205 with SMTP id dm13mr10217462obb.9.1352510369650; Fri, 09 Nov 2012 17:19:29 -0800 (PST) Received: by 10.76.143.33 with HTTP; Fri, 9 Nov 2012 17:19:29 -0800 (PST) Date: Fri, 9 Nov 2012 17:19:29 -0800 Message-ID: Subject: Infiniband list available From: Garrett Cooper To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-infiniband@freebsd.org X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Nov 2012 01:19:33 -0000 Hi all, With the help of postmaster, a new Infiniband list has been created for FreeBSD. If you are interested in IB on FreeBSD, please subscribe to the list. My group (Isilon) will be providing more details about the porting effort being done from our end to the latest 1.5.x and the group is working at improving IB documentation and the like in the interim. Feel free to discuss anything IB related on the list! Cheers! -Garrett From owner-freebsd-infiniband@FreeBSD.ORG Sat Nov 10 04:13:05 2012 Return-Path: Delivered-To: freebsd-infiniband@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 89AD2420 for ; Sat, 10 Nov 2012 04:13:05 +0000 (UTC) (envelope-from lists@jnielsen.net) Received: from ns1.jnielsen.net (secure.freebsdsolutions.net [69.55.234.48]) by mx1.freebsd.org (Postfix) with ESMTP id 4B7B18FC12 for ; Sat, 10 Nov 2012 04:13:04 +0000 (UTC) Received: from [192.168.2.82] (c-67-182-217-170.hsd1.ut.comcast.net [67.182.217.170]) (authenticated bits=0) by ns1.jnielsen.net (8.14.4/8.14.4) with ESMTP id qAA3icf5001038 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT) for ; Fri, 9 Nov 2012 22:44:39 -0500 (EST) (envelope-from lists@jnielsen.net) From: John Nielsen Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: Fwd: using ConnectX card as Ethernet (mlxen) Date: Fri, 9 Nov 2012 20:44:36 -0700 References: <3A359B33-380C-4230-A62C-623765E9376A@jnielsen.net> To: freebsd-infiniband@freebsd.org Message-Id: Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) X-Mailer: Apple Mail (2.1499) X-DCC-sonic.net-Metrics: ns1.jnielsen.net 1117; Body=1 Fuz1=1 Fuz2=1 X-Virus-Scanned: clamav-milter 0.97.5 at ns1.jnielsen.net X-Virus-Status: Clean X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Nov 2012 04:13:05 -0000 On the off chance there's someone on this list already that didn't see = my earlier (unanswered) question, I'm reposting it below. I don't have = FreeBSD on this hardware any more but I would like to know for future = reference if using these cards with an Ethernet switch is an option. Begin forwarded message: > From: John Nielsen > Subject: using ConnectX card as Ethernet (mlxen) > Date: September 24, 2012 10:37:30 AM MDT > To: freebsd-current@freebsd.org > Cc: Jeff Roberson >=20 > I have a machine running "FreeBSD 10.0-CURRENT #0 r240887" amd64 with = two ConnectX (InfiniBand) cards. Relevant bits of dmesg and pciconf -lv = below. The cards are connected directly to a 10GB Ethernet switch so I = need to run them in "eth" mode rather than "ib". Unfortunately they come = up in "ib" mode and I don't know how to change it. >=20 > The same hardware works fine under CentOS 6.3, though I need to = manually set the cards to 'eth' there as well (which I do using a = 'connectx_port_config script from Mellanox that twiddles the mlx4_port1 = entries under /sys (sysfs). Under FreeBSD I see these sysctls but I = can't set them to 'eth' either via /boot/loader.conf or by sysctl after = boot, with or without mlxen and/or mlx4ib loaded: > sys.device.mlx4_core0.mlx4_port1: ib > sys.device.mlx4_core1.mlx4_port1: ib >=20 > Assuming mlxen is actually supported, how do I configure the card so = it will attach? >=20 >=20 > mlx4_core0: mem = 0xdfa00000-0xdfafffff,0xdd800000-0xddffffff irq 32 at device 0.0 on pci4 > mlx4_core: Mellanox ConnectX core driver v1.0-ofed1.5.2 (August 4, = 2010) > mlx4_core: Initializing mlx4_core > mlx4_en: Mellanox ConnectX HCA Ethernet driver v1.5.2 (July 2010) > mlx4_en mlx4_core0: UDP RSS is not supported on this device. > mlx4_core1: mem = 0xdf900000-0xdf9fffff,0xdd000000-0xdd7fffff irq 42 at device 0.0 on pci7 > mlx4_core: Initializing mlx4_core >=20 > mlx4_core0@pci0:4:0:0: class=3D0x0c0600 card=3D0x002215b3 = chip=3D0x673c15b3 rev=3D0xb0 hdr=3D0x00 > vendor =3D 'Mellanox Technologies' > device =3D 'MT26428 [ConnectX VPI PCIe 2.0 5GT/s - IB QDR / = 10GigE]' > class =3D serial bus > mlx4_core1@pci0:7:0:0: class=3D0x028000 card=3D0x001715b3 = chip=3D0x100315b3 rev=3D0x00 hdr=3D0x00 > vendor =3D 'Mellanox Technologies' > device =3D 'MT27500 Family [ConnectX-3]' > class =3D network >=20 > Thanks, >=20 > JN >=20 From owner-freebsd-infiniband@FreeBSD.ORG Sat Nov 10 04:51:36 2012 Return-Path: Delivered-To: freebsd-infiniband@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 007D495A for ; Sat, 10 Nov 2012 04:51:35 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-oa0-f54.google.com (mail-oa0-f54.google.com [209.85.219.54]) by mx1.freebsd.org (Postfix) with ESMTP id A81F98FC12 for ; Sat, 10 Nov 2012 04:51:35 +0000 (UTC) Received: by mail-oa0-f54.google.com with SMTP id n9so5907230oag.13 for ; Fri, 09 Nov 2012 20:51:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=16JXdS3hyJl2Jfg3y3P4Ga6Cc4ebynCbcIZ1Jn5L/vk=; b=Bf2052HZVHofc7y1TUVyugX2DTCtSluHo+yf5NKAe1zkPEFdTugzcWS2FwDLmBRHln Fic5eu55uvtQ3KAM/4GIz+FPtXM6/oXwqxMN//FsWu/2UijGCarb8t2o7XQNluMjeM5d elkh6ySWJQhDKmSqTe6l21iskOgHUQRGws5iEppaRz1SX71f+/dB9y1wpmMoCA6vUHwt aLmO2hyk8ru4cIhENE3kY7kTw2YNyw7dnaNy4OB7MkWMQfMI8o+exHXV/asILiFzWQYB /jrzJvPUn7giJibxkQcABLBm1N5+FykrVVLJzMpclmFrJ4RQen9fBbmEpxTdPB1afqnd rPUg== MIME-Version: 1.0 Received: by 10.182.197.8 with SMTP id iq8mr2263137obc.66.1352523094649; Fri, 09 Nov 2012 20:51:34 -0800 (PST) Received: by 10.76.143.33 with HTTP; Fri, 9 Nov 2012 20:51:34 -0800 (PST) In-Reply-To: References: <3A359B33-380C-4230-A62C-623765E9376A@jnielsen.net> Date: Fri, 9 Nov 2012 20:51:34 -0800 Message-ID: Subject: Re: using ConnectX card as Ethernet (mlxen) From: Garrett Cooper To: John Nielsen Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-infiniband@freebsd.org X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Nov 2012 04:51:36 -0000 On Fri, Nov 9, 2012 at 7:44 PM, John Nielsen wrote: > On the off chance there's someone on this list already that didn't see my > earlier (unanswered) question, I'm reposting it below. I don't have FreeBSD > on this hardware any more but I would like to know for future reference if > using these cards with an Ethernet switch is an option. > I know this might seem dumb, but what if you did sys.device.mlx4_core0.mlx4_port1="`printf 'en\n'`" (the newline's important). Thanks! -Garrett