Date: Fri, 9 Nov 2012 20:44:36 -0700 From: John Nielsen <lists@jnielsen.net> To: freebsd-infiniband@freebsd.org Subject: Fwd: using ConnectX card as Ethernet (mlxen) Message-ID: <FB2BC2D5-46CC-4CF6-8EC8-5C2836253D14@jnielsen.net> References: <3A359B33-380C-4230-A62C-623765E9376A@jnielsen.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On the off chance there's someone on this list already that didn't see my earlier (unanswered) question, I'm reposting it below. I don't have FreeBSD on this hardware any more but I would like to know for future reference if using these cards with an Ethernet switch is an option. Begin forwarded message: > From: John Nielsen <lists@jnielsen.net> > Subject: using ConnectX card as Ethernet (mlxen) > Date: September 24, 2012 10:37:30 AM MDT > To: freebsd-current@freebsd.org > Cc: Jeff Roberson <jeff@freebsd.org> > > I have a machine running "FreeBSD 10.0-CURRENT #0 r240887" amd64 with two ConnectX (InfiniBand) cards. Relevant bits of dmesg and pciconf -lv below. The cards are connected directly to a 10GB Ethernet switch so I need to run them in "eth" mode rather than "ib". Unfortunately they come up in "ib" mode and I don't know how to change it. > > The same hardware works fine under CentOS 6.3, though I need to manually set the cards to 'eth' there as well (which I do using a 'connectx_port_config script from Mellanox that twiddles the mlx4_port1 entries under /sys (sysfs). Under FreeBSD I see these sysctls but I can't set them to 'eth' either via /boot/loader.conf or by sysctl after boot, with or without mlxen and/or mlx4ib loaded: > sys.device.mlx4_core0.mlx4_port1: ib > sys.device.mlx4_core1.mlx4_port1: ib > > Assuming mlxen is actually supported, how do I configure the card so it will attach? > > > mlx4_core0: <mlx4_core> mem 0xdfa00000-0xdfafffff,0xdd800000-0xddffffff irq 32 at device 0.0 on pci4 > mlx4_core: Mellanox ConnectX core driver v1.0-ofed1.5.2 (August 4, 2010) > mlx4_core: Initializing mlx4_core > mlx4_en: Mellanox ConnectX HCA Ethernet driver v1.5.2 (July 2010) > mlx4_en mlx4_core0: UDP RSS is not supported on this device. > mlx4_core1: <mlx4_core> mem 0xdf900000-0xdf9fffff,0xdd000000-0xdd7fffff irq 42 at device 0.0 on pci7 > mlx4_core: Initializing mlx4_core > > mlx4_core0@pci0:4:0:0: class=0x0c0600 card=0x002215b3 chip=0x673c15b3 rev=0xb0 hdr=0x00 > vendor = 'Mellanox Technologies' > device = 'MT26428 [ConnectX VPI PCIe 2.0 5GT/s - IB QDR / 10GigE]' > class = serial bus > mlx4_core1@pci0:7:0:0: class=0x028000 card=0x001715b3 chip=0x100315b3 rev=0x00 hdr=0x00 > vendor = 'Mellanox Technologies' > device = 'MT27500 Family [ConnectX-3]' > class = network > > Thanks, > > JN >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?FB2BC2D5-46CC-4CF6-8EC8-5C2836253D14>