From owner-freebsd-infiniband@freebsd.org Fri Mar 16 16:44:50 2018 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8E7D0F5CD4A; Fri, 16 Mar 2018 16:44:50 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (turbocat.net [IPv6:2a01:4f8:c17:6c4b::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id ED7E268A27; Fri, 16 Mar 2018 16:44:49 +0000 (UTC) (envelope-from hps@selasky.org) Received: from hps2016.home.selasky.org (unknown [62.141.128.70]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 857B12603AE; Fri, 16 Mar 2018 17:44:47 +0100 (CET) Subject: Re: [HEADS UP] - OFED/RDMA stack update To: Konstantin Belousov , Navdeep Parhar Cc: "'freebsd-infiniband@freebsd.org'" , freebsd-drivers , Meny Yossefi , "'FreeBSD-stable@FreeBSD.org'" , freebsd-arch References: <1519683699.47932.5.camel@FreeBSD.org> <20180226224311.GT94212@kib.kiev.ua> From: Hans Petter Selasky Message-ID: <3027f48e-0ba8-555d-df23-d638303cb125@selasky.org> Date: Fri, 16 Mar 2018 17:44:41 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: <20180226224311.GT94212@kib.kiev.ua> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Mar 2018 16:44:50 -0000 Hi, The bsd_rdma_4_9_stable_11 projects branch is close to being merged into FreeBSD 11-stable. Mellanox plans to merge no later than 12:00 CEST TUE 20th of March 2018, unless objections are received. A compatibility header file has been created, ib_verbs_compat.h, which offers full source compatibility to existing OFED kernel applications, as a response to the raised conserns. User-space compatibility is maintained through library symbol versioning. https://svnweb.freebsd.org/base/projects/bsd_rdma_4_9_stable_11/sys/ofed/include/rdma/ib_verbs_compat.h An example client for this header file can be found here: https://svnweb.freebsd.org/base/projects/bsd_rdma_4_9_stable_11/sys/contrib/rdma/krping_compat/ Currently the cxgb and cxgbe i-Warp drivers are not building, and will be stubbed from the kernel build before the branch is merged, unless Chelsio can add patches for these. Here is a quick and dirty patch to make the bsd_rdma_4_9_stable_11 branch build: > diff --git a/sys/modules/Makefile b/sys/modules/Makefile > index 6b005c854d7..b918a208f21 100644 > --- a/sys/modules/Makefile > +++ b/sys/modules/Makefile > @@ -530,7 +530,7 @@ _txp= txp > .if ${MK_SOURCELESS_UCODE} != "no" && ${MACHINE_CPUARCH} != "arm" && \ > ${MACHINE_ARCH:C/mips(el)?/mips/} != "mips" && \ > ${MACHINE_ARCH} != "powerpc" && ${MACHINE_CPUARCH} != "riscv" > -_cxgbe= cxgbe > +#_cxgbe= cxgbe > .endif > > .if ${MK_TESTS} != "no" || defined(ALL_MODULES) > @@ -554,7 +554,7 @@ _vpo= vpo > _sym= sym > # intr_disable() is a macro, causes problems > .if ${MK_SOURCELESS_UCODE} != "no" > -_cxgb= cxgb > +#_cxgb= cxgb > .endif > .endif --HPS From owner-freebsd-infiniband@freebsd.org Sat Mar 17 19:52:36 2018 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DF4D8F64340; Sat, 17 Mar 2018 19:52:35 +0000 (UTC) (envelope-from nparhar@gmail.com) Received: from mail-pl0-x243.google.com (mail-pl0-x243.google.com [IPv6:2607:f8b0:400e:c01::243]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5CA6A6BD12; Sat, 17 Mar 2018 19:52:35 +0000 (UTC) (envelope-from nparhar@gmail.com) Received: by mail-pl0-x243.google.com with SMTP id m22-v6so7845160pls.5; Sat, 17 Mar 2018 12:52:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:mail-followup-to :references:mime-version:content-disposition:in-reply-to:user-agent; bh=WOzsTN7kJIZXhrUPU7HNhxhkX4eeYLTa2xaccMXJujQ=; b=BVZJ9zowhwLecc6zkDlWSd+Tq4NTIyuPncmeT3XTOwl+cVEdodCZcG8DywxQ2ir/qz XO0Bq6VGM5jpl9PcvzY832ZYq7rPgg+DqNbjvDPZn7Q61EJaueD8YaRDZJQXV1DZRC7g c8aOtZcSRWCNBeuNoH4q7P2BdMgfkQNrxLalyFfeJgeansfXshGJgO9ahP5BObes3pbF 5v3idzqy/sku37j1p5CvFK2WSMVtMCobMljtPmf41tgEO0j2PzCkXg0S9ylVUN2VG2w0 WzJWfbjAQR06GggY+4MQdkBjgb3fZWl7a9ETAB8TEdNqzGO8qkUDCovuXVQScvmypQ6V gCng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :mail-followup-to:references:mime-version:content-disposition :in-reply-to:user-agent; bh=WOzsTN7kJIZXhrUPU7HNhxhkX4eeYLTa2xaccMXJujQ=; b=cFPJAmt4cLcMDW1ZOhV/vtt2goyFYfiz7klzQ2GZo0l8VGi5xYEhbrh/bfuJ62xLJx JQkP6lumPIZ3o8Nmce4qP4oQb6BVXyCF+eNdiUcNLp6ZdRFRDBn0Ka5cwPWS0RNvx2dk MjnNXYXfEaDQamfzOwnbV4AUySBkH4fHBldBSSFbrhDitBfwruD1PDvFHO56jyYXcmYq 0lMzD/Bw+I5oDmHNeL1rYH6IagktqcfIi+HsAnBdbsBTFTkpezWPeYJbrMqoMi0aaIps pdHCK92QAfRyAOVsxvamY5Q79GuP97CRZWa6E+Uyg7LGkJJM87oNzD+e+d9v/ckyYlt2 mNfg== X-Gm-Message-State: AElRT7EWWf/w6LkpXZ+Hn4smSPrS6+4XaSs/tgQ9+Kdf2kqlL/ikLGaY EYD1kceNcfqef1/AjwR9908bfvNT X-Google-Smtp-Source: AG47ELs97s0plY6rCVbH/YAcdfo6b7Ua0WSTdAV0Skj5tzbDeuc1GwaXEoE4CENw1CdI/yo/MILccQ== X-Received: by 2002:a17:902:108a:: with SMTP id c10-v6mr1466917pla.22.1521316354379; Sat, 17 Mar 2018 12:52:34 -0700 (PDT) Received: from ox ([2601:641:c000:b800:c55e:d964:7a77:a05]) by smtp.gmail.com with ESMTPSA id v6sm20058516pfm.2.2018.03.17.12.52.33 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 17 Mar 2018 12:52:33 -0700 (PDT) Sender: Navdeep Parhar Date: Sat, 17 Mar 2018 12:52:16 -0700 From: Navdeep Parhar To: Hans Petter Selasky Cc: Konstantin Belousov , "'freebsd-infiniband@freebsd.org'" , freebsd-drivers , Meny Yossefi , "'FreeBSD-stable@FreeBSD.org'" , freebsd-arch Subject: Re: [HEADS UP] - OFED/RDMA stack update Message-ID: <20180317195200.GA5223@ox> Mail-Followup-To: Hans Petter Selasky , Konstantin Belousov , "'freebsd-infiniband@freebsd.org'" , freebsd-drivers , Meny Yossefi , "'FreeBSD-stable@FreeBSD.org'" , freebsd-arch References: <1519683699.47932.5.camel@FreeBSD.org> <20180226224311.GT94212@kib.kiev.ua> <3027f48e-0ba8-555d-df23-d638303cb125@selasky.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3027f48e-0ba8-555d-df23-d638303cb125@selasky.org> User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Mar 2018 19:52:36 -0000 Hold your horses. Do you have confirmation from the affected party that the shims are adequate for them? I have been waiting for that before looking at this branch. Is the iw_cxgbe breakage a simple merge conflict as previously discussed or do the shims require driver changes? If they don't then it should be possible to drop the iw_cxgbe from head into this branch and it should just work, is that correct? Regards, Navdeep On Fri, Mar 16, 2018 at 05:44:41PM +0100, Hans Petter Selasky wrote: > Hi, > > The bsd_rdma_4_9_stable_11 projects branch is close to being merged into > FreeBSD 11-stable. Mellanox plans to merge no later than 12:00 CEST TUE 20th > of March 2018, unless objections are received. > > A compatibility header file has been created, ib_verbs_compat.h, which > offers full source compatibility to existing OFED kernel applications, as a > response to the raised conserns. User-space compatibility is maintained > through library symbol versioning. > > https://svnweb.freebsd.org/base/projects/bsd_rdma_4_9_stable_11/sys/ofed/include/rdma/ib_verbs_compat.h > > An example client for this header file can be found here: > > https://svnweb.freebsd.org/base/projects/bsd_rdma_4_9_stable_11/sys/contrib/rdma/krping_compat/ > > Currently the cxgb and cxgbe i-Warp drivers are not building, and will be > stubbed from the kernel build before the branch is merged, unless Chelsio > can add patches for these. > > Here is a quick and dirty patch to make the bsd_rdma_4_9_stable_11 branch > build: > > >diff --git a/sys/modules/Makefile b/sys/modules/Makefile > >index 6b005c854d7..b918a208f21 100644 > >--- a/sys/modules/Makefile > >+++ b/sys/modules/Makefile > >@@ -530,7 +530,7 @@ _txp= txp > > .if ${MK_SOURCELESS_UCODE} != "no" && ${MACHINE_CPUARCH} != "arm" && \ > > ${MACHINE_ARCH:C/mips(el)?/mips/} != "mips" && \ > > ${MACHINE_ARCH} != "powerpc" && ${MACHINE_CPUARCH} != "riscv" > >-_cxgbe= cxgbe > >+#_cxgbe= cxgbe > > .endif > > .if ${MK_TESTS} != "no" || defined(ALL_MODULES) > >@@ -554,7 +554,7 @@ _vpo= vpo > > _sym= sym > > # intr_disable() is a macro, causes problems > > .if ${MK_SOURCELESS_UCODE} != "no" > >-_cxgb= cxgb > >+#_cxgb= cxgb > > .endif > > .endif > > --HPS From owner-freebsd-infiniband@freebsd.org Sat Mar 17 20:03:48 2018 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 036A2F64F74; Sat, 17 Mar 2018 20:03:48 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (turbocat.net [IPv6:2a01:4f8:c17:6c4b::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8E9376C3DB; Sat, 17 Mar 2018 20:03:47 +0000 (UTC) (envelope-from hps@selasky.org) Received: from hps2016.home.selasky.org (unknown [62.141.128.70]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 98064260370; Sat, 17 Mar 2018 21:03:45 +0100 (CET) Subject: Re: [HEADS UP] - OFED/RDMA stack update To: Konstantin Belousov , "'freebsd-infiniband@freebsd.org'" , freebsd-drivers , Meny Yossefi , "'FreeBSD-stable@FreeBSD.org'" , freebsd-arch References: <1519683699.47932.5.camel@FreeBSD.org> <20180226224311.GT94212@kib.kiev.ua> <3027f48e-0ba8-555d-df23-d638303cb125@selasky.org> <20180317195200.GA5223@ox> From: Hans Petter Selasky Message-ID: <6d451a3b-a635-08b1-f8d4-52fdc48083d6@selasky.org> Date: Sat, 17 Mar 2018 21:03:40 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: <20180317195200.GA5223@ox> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Mar 2018 20:03:48 -0000 On 03/17/18 20:52, Navdeep Parhar wrote: > Hold your horses. Do you have confirmation from the affected party that > the shims are adequate for them? I have been waiting for that before > looking at this branch. Hi Navdeep, Mellanox has received an API list from at least one party, and has taken the action to support all the required APIs. > Is the iw_cxgbe breakage a simple merge conflict as previously discussed > or do the shims require driver changes? It is a merge conflict. The code already compiles in 12-current. > If they don't then it should be > possible to drop the iw_cxgbe from head into this branch and it should > just work, is that correct? Yes, it shouldn't be too hard to do and I would appreciate if Chelsio could throw some resources at this ASAP. I believe the new code will benefit everyone using Infiniband/RoCE and iWarp with FreeBSD. --HPS