Date: Mon, 11 Feb 2013 12:06:41 -0800 From: Tamer Abuelata <tamer@polycoresoftware.com> To: freebsd-infiniband@freebsd.org Subject: OFED setup and Mellanox cards Message-ID: <33AE1DB1-7BA9-4329-8AF0-EC7FEDCFBE7F@polycoresoftware.com>
next in thread | raw e-mail | index | archive | help
Hi, I'm trying to get OFED up and running on a FreeBSD 9.1 machine with 2 Mellanox cards (MT26428). We have compiled the Kernel with the following options (per https://wiki.freebsd.org/InfiniBand ): options OFED # Infiniband protocol stack and support options SDP # Sockets Direct Protocol for infiniband options IPOIB_CM # Use connect mode ipoib device ipoib # IP over IB devices device mlx4ib # ConnectX Infiniband support device mlxen # ConnectX Ethernet support device mthca # Infinihost cards I gave an IP to each card. # ifconfig ib0: flags=8043<UP,BROADCAST,RUNNING,MULTICAST> metric 0 mtu 65520 options=80018<VLAN_MTU,VLAN_HWTAGGING,LINKSTATE> lladdr 80.0.0.48.fe.80.0.0.0.0.0.0.0.25.90.ff.ff.7.70.d9 inet 10.0.0.1 netmask 0xffffff00 broadcast 10.0.0.255 nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> ib1: flags=8043<UP,BROADCAST,RUNNING,MULTICAST> metric 0 mtu 65520 options=80018<VLAN_MTU,VLAN_HWTAGGING,LINKSTATE> lladdr 80.0.0.48.fe.80.0.0.0.0.0.0.0.25.90.ff.ff.7.70.f1 inet 10.0.0.2 netmask 0xffffff00 broadcast 10.0.0.255 nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> After doing a build world and build kernel I try to run an example application and get the following warnings/errors: # ./server libibverbs: Warning: no userspace device-specific driver found for /sys/class/infiniband_verbs/uverbs1 libibverbs: Warning: no userspace device-specific driver found for /sys/class/infiniband_verbs/uverbs0 libibverbs: Warning: no userspace device-specific driver found for /sys/class/infiniband_verbs/uverbs1 libibverbs: Warning: no userspace device-specific driver found for /sys/class/infiniband_verbs/uverbs0 libibverbs: Warning: no userspace device-specific driver found for /sys/class/infiniband_verbs/uverbs1 libibverbs: Warning: no userspace device-specific driver found for /sys/class/infiniband_verbs/uverbs0 listening on port 29823. Then I launch the client from another terminal: # ./client 10.0.0.2 29823 libibverbs: Warning: no userspace device-specific driver found for /sys/class/infiniband_verbs/uverbs1 libibverbs: Warning: no userspace device-specific driver found for /sys/class/infiniband_verbs/uverbs0 libibverbs: Warning: no userspace device-specific driver found for /sys/class/infiniband_verbs/uverbs1 libibverbs: Warning: no userspace device-specific driver found for /sys/class/infiniband_verbs/uverbs0 [… shows several more times …] Segmentation fault (core dumped) So the sample app compiles fine but I get this runtime error which seems to come from the libibverbs. This /sys/class/infiniband_verbs folder does not exist. I'm not sure if I'm missing configuration files. However I have devices at /dev/uverbs0 and /dev/uverbs1. The example app I am using is from: http://thegeekinthecorner.wordpress.com/2010/08/14/building-an-rdma-capable-application-with-ib-verbs-part-2-the-server/ The code is linked at the bottom of the page. Direct link: https://sites.google.com/a/bedeir.com/home/basic-rdma-client-server.tar.gz?attredirects=0&d=1 I would appreciate any help or pointers from someone who managed to get OFED to run on FreeBSD. Best regards, Tamer
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?33AE1DB1-7BA9-4329-8AF0-EC7FEDCFBE7F>