Date: Wed, 27 Apr 2016 11:54:01 -0500 From: Jason Bacon <bacon@uwm.edu> To: Chris Reeves <chrisr@spectralogic.com>, "freebsd-infiniband@freebsd.org" <freebsd-infiniband@freebsd.org> Subject: Re: OFED v3.2 update Message-ID: <5720EEA9.6030203@uwm.edu> In-Reply-To: <1459534095550.93717@spectralogic.com> References: <56FCF309.1070406@selasky.org> <1459534095550.93717@spectralogic.com>
next in thread | previous in thread | raw e-mail | index | archive | help
This is encouraging... On our CentOS 6.7 cluster, what we're seeing is comparable to the Ubuntu performance you reported earlier. ( Best of 3 runs between two idle compute nodes on the same leaf switch shown below. ) It appears that we may be able to improve NFS performance using a FreeBSD/FreeNAS file server in the near future. Regards, Jason Linux compute-083. bacon ~ 335: iperf -c compute-082-ib0 ------------------------------------------------------------ Client connecting to compute-082-ib0, TCP port 5001 TCP window size: 645 KByte (default) ------------------------------------------------------------ [ 3] local 10.2.11.109 port 47470 connected with 10.2.11.108 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.0 sec 22.3 GBytes 19.2 Gbits/sec On 04/01/16 13:08, Chris Reeves wrote: > Thank you Hans, > > Using these patches, I was able to get my IPOIB performance up to 25 Gb/s (vs 10 Gb/s on 11-current without the patches). > > Out of curiosity, do you have any plans to MFC these to stable-10? > > Thanks, > Chris > ________________________________________ > From: owner-freebsd-infiniband@freebsd.org <owner-freebsd-infiniband@freebsd.org> on behalf of Hans Petter Selasky <hps@selasky.org> > Sent: Thursday, March 31, 2016 3:51 AM > To: freebsd-infiniband@freebsd.org > Subject: OFED v3.2 update > > Dear users of infiniband in FreeBSD: > > > Mellanox is working on a big infiniband update towards mellanox OFED > v3.2 of the infiniband stack (ibcore), drivers (mlx4/mlx5) and libraries > (librdma, libibverbs ...). Currently there are 10-patches in different > areas of the FreeBSD kernel and userspace which you need to apply to > 11-current as of today to get a working GENERIC kernel. > > Compile world and kernel like this after applying the patches below: > > make buildkernel WITH_OFED=YES > make buildworld WITH_OFED=YES > > > https://reviews.freebsd.org/D5790 > https://reviews.freebsd.org/D5791 > https://reviews.freebsd.org/D5792 > https://reviews.freebsd.org/D5793 > https://reviews.freebsd.org/D5794 > https://reviews.freebsd.org/D5795 > https://reviews.freebsd.org/D5796 > https://reviews.freebsd.org/D5797 > https://reviews.freebsd.org/D5798 > https://reviews.freebsd.org/D5799 > > > The patches are work in progress, though at a level where basic > infiniband is working with MLX5. > > > Known issues: > - LINT build is broken (Will be fixed before committing. Waiting for > solution from Navdeep) > - Need to unload ipoib before mlx5ib. > > --HPS > _______________________________________________ > freebsd-infiniband@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-infiniband > To unsubscribe, send any mail to "freebsd-infiniband-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-infiniband@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-infiniband > To unsubscribe, send any mail to "freebsd-infiniband-unsubscribe@freebsd.org" -- ------------------------------------- Jason W. Bacon Systems Manager Research Computing Support University of Wisconsin Milwaukee bacon@uwm.edu -------------------------------------
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5720EEA9.6030203>