From owner-freebsd-infiniband@freebsd.org Mon Feb 26 14:50:06 2018 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AFFE4F2A1D5; Mon, 26 Feb 2018 14:50:06 +0000 (UTC) (envelope-from menyy@mellanox.com) Received: from EUR03-VE1-obe.outbound.protection.outlook.com (mail-eopbgr50089.outbound.protection.outlook.com [40.107.5.89]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT TLS CA 4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 17D047841A; Mon, 26 Feb 2018 14:50:02 +0000 (UTC) (envelope-from menyy@mellanox.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=PCqD+BIK2GD2QhFztH9uztskEKWses6fleYzlS3tkkY=; b=YPSg9GUPDSnhqTmx1ZtqpL4POl0LjYoBVjF9Rfc992Ku83L9Ui6FFLUtnu3GKeybUA58dE//f6E96Mo/zAhQKRoBOLPA8xrjm19r+WjSrqIZsKH2ZE7b/bDOcpFISocJvyrklrVy82kVWmm4eI7zh9eEXhaJRz5eBZlZlZ9VlLQ= Received: from VI1PR0501MB2863.eurprd05.prod.outlook.com (10.172.12.8) by VI1PR0501MB2063.eurprd05.prod.outlook.com (10.167.195.153) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.527.15; Mon, 26 Feb 2018 14:49:58 +0000 Received: from VI1PR0501MB2863.eurprd05.prod.outlook.com ([fe80::89b0:b1d8:fec6:c6ec]) by VI1PR0501MB2863.eurprd05.prod.outlook.com ([fe80::89b0:b1d8:fec6:c6ec%17]) with mapi id 15.20.0527.021; Mon, 26 Feb 2018 14:49:58 +0000 From: Meny Yossefi To: "'freebsd-infiniband@freebsd.org'" , "'FreeBSD-stable@FreeBSD.org'" CC: freebsd-drivers Subject: RE: [HEADS UP] - OFED/RDMA stack update Thread-Topic: [HEADS UP] - OFED/RDMA stack update Thread-Index: AdNZQQ/RuIbNiZgtQumqRm3gP9gf8RVzyJuw Date: Mon, 26 Feb 2018 14:49:58 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=menyy@mellanox.com; x-originating-ip: [193.47.165.251] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; VI1PR0501MB2063; 7:C/aNGuhb+8Yr1y6azoNoMhTDGZGVRLwsAi9THdtseggsEohmY4mdU6rOaYeTYMMOi89eKR3r5YtjfIAg1HSjM5EhZ/Pr1y2Wl9LOZ6VxcwFNiMFBpkNgHoeg/kLWpDD9HFxjOcCXjyPkAwV4sXEbUfCtjepiFN/fITtPihT1SNHPNvtuHIaukhhJPbhu3buk3T/4TGL7nbYpt2EJn9f+z8S4JBgpGHl0EqHWnQ2RlPZvaQv0B9AbyMdeedKV1yjD x-ms-exchange-antispam-srfa-diagnostics: SSOS; x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: d99f9c42-bdfc-497e-929d-08d57d28354a x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(48565401081)(4534165)(4627221)(201703031133081)(201702281549075)(5600026)(4604075)(3008032)(2017052603307)(7153060)(7193020); SRVR:VI1PR0501MB2063; x-ms-traffictypediagnostic: VI1PR0501MB2063: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(5213294742642); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040501)(2401047)(5005006)(8121501046)(10201501046)(93006095)(93001095)(3231220)(944501161)(52105095)(3002001)(6055026)(6041288)(20161123564045)(20161123562045)(20161123560045)(20161123558120)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(6072148)(201708071742011); SRVR:VI1PR0501MB2063; BCL:0; PCL:0; RULEID:; SRVR:VI1PR0501MB2063; x-forefront-prvs: 05954A7C45 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(346002)(366004)(396003)(39860400002)(376002)(39380400002)(199004)(189003)(13464003)(86362001)(3660700001)(33656002)(3846002)(6116002)(5250100002)(450100002)(229853002)(3280700002)(102836004)(99286004)(14454004)(105586002)(305945005)(74316002)(106356001)(6436002)(53546011)(68736007)(6506007)(2900100001)(7736002)(7696005)(25786009)(6246003)(478600001)(8676002)(81166006)(97736004)(81156014)(66066001)(2906002)(53936002)(4743002)(186003)(59450400001)(316002)(5660300001)(8936002)(110136005)(107886003)(26005)(9686003)(15650500001)(55016002)(4326008)(96633001)(491001); DIR:OUT; SFP:1101; SCL:1; SRVR:VI1PR0501MB2063; H:VI1PR0501MB2863.eurprd05.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: 6SwSGUyXJhQmB7A1hoZrX43luTVBRikRn47WkEAtrfRkojQK+tZNmrt1NfmWlkU4Fpmiyg2TqEo3KDy32Q8DueZE/2RZIBvSpOnAAOLRtrC0Cc/xVqt3VVbUl+amThbY9v8gZ0hdo0snhq74BvwLVkAHdw650NwqUbvdnoPMzCs= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: d99f9c42-bdfc-497e-929d-08d57d28354a X-MS-Exchange-CrossTenant-originalarrivaltime: 26 Feb 2018 14:49:58.7786 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0501MB2063 X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Feb 2018 14:50:06 -0000 Hi,=20 We are currently working on MFC'ing the OFED/RDMA stack update mentioned be= low to FreeBSD11-STABLE. We already have working version in 'projects/bsd_rdma_4_9_stable_11' (pendi= ng iWARP updates) and currently working on ULP integration. Again, as always, for any concern/comments you might have, please don't hes= itate contacting us. freebsd-drivers@mellanox.com Regards,=20 Meny Yossefi,=20 Mellanox technologies -----Original Message----- From: Meny Yossefi=20 Sent: Monday, November 13, 2017 11:09 AM To: 'freebsd-infiniband@freebsd.org' ; 'fre= ebsd-current@freebsd.org' Cc: freebsd-drivers Subject: [HEADS UP] - OFED/RDMA stack update=20 Hi,=20 This is to inform you that by end of this week we plan to merge the OFED/RD= MA stack update from the project - 'bsd_rdma_4_9' into 12-CURRENT. The update aligns the OFED common code and RDMA vendor drivers with Linux v= 4.9. We are still working on final modifications and build testing it prior to s= ubmission. For any concern/comments you might have, please don't hesitate contacting u= s. freebsd-drivers@mellanox.com Regards,=20 Meny Yossefi,=20 Mellanox technologies From owner-freebsd-infiniband@freebsd.org Mon Feb 26 22:21:48 2018 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C736CF27D47; Mon, 26 Feb 2018 22:21:48 +0000 (UTC) (envelope-from nparhar@gmail.com) Received: from mail-wr0-x232.google.com (mail-wr0-x232.google.com [IPv6:2a00:1450:400c:c0c::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3A7C96D61D; Mon, 26 Feb 2018 22:21:45 +0000 (UTC) (envelope-from nparhar@gmail.com) Received: by mail-wr0-x232.google.com with SMTP id m5so22864888wrg.1; Mon, 26 Feb 2018 14:21:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:message-id:subject:from:to:cc:date:in-reply-to:references :mime-version:content-transfer-encoding; bh=7TEoEkKjF6un0YpEEQyLNTPUSN2yMd4r1fJ9DUhImI4=; b=BY0DVDXg9TTOe33IW6ld49mJMYBXtn1NQcXC6epwAPfAMexeWzB1E/aCngOcjtP3o5 PscU8MorYlimJpfpuQc0cTY3PYDT0EXlT3U4rW7iTEB6vU1ZzBpckUNZ91ikqI65WC0y Z/o+pDVjAkg9XJyvav0+WFBMFXvDrn4c1JtxtM1L/JDmwqM7pxHGyY1qXEKUZacdR5rD mpGZfIan1Aym79DwSHip//BC1LIopWl7OaTVIMpC2Ne20MhQOmn7OqQePuAmHOcRMpOC 2XytGVTlGaNYGsdbs5EjIfHMUaFXfTJSe8EcCyUbFcO8FQ+MT0Dcgx4FPR1EDx41OU8J wbjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:message-id:subject:from:to:cc:date :in-reply-to:references:mime-version:content-transfer-encoding; bh=7TEoEkKjF6un0YpEEQyLNTPUSN2yMd4r1fJ9DUhImI4=; b=kxq+hMTT/cuKKzTMoTrx0m97tGRvvrwle9UIDQZ+CRASkjXwNS2PN8n59r5H0YJOrP /H5oWCtMEHzf5al8/goA6ZHhlMIvt6uGBM7Y6iXpxpL8LyuFZLhKwNBvirVd+vsHxuKE sLzQCuGq3s7sVjjowIJC7Ohr/elvnqEgtKLfJLPrfOy4J7vQZglMALmGL+eO3s9d0sHD tFht5Okj9JcIRzS8334J0z1zPtSNpZic3w3Fnd7RMzqtIovjohmLWEsjldtWcS45KC9b aPkXCXYgihH7OYmUqRHS/LPShtHwPop28BcMiOY0uHumuC3ykHpHJOx4xkANLdFhFXM0 FadA== X-Gm-Message-State: APf1xPBxU5S+oYZTYmSY/CJcQawBefZVj2c1gqDvrF2PeaVu5CyC+TZl N9Y+BaFNy4I8qnkGbR7ibHQ= X-Google-Smtp-Source: AH8x226LoVJEauXUt4YqcUbLSOPMj9wYZwyu5yBg3oVrVapL2tH6I0Vad2kp1lAHSjqJ3xKphJhnOA== X-Received: by 10.223.166.171 with SMTP id t40mr11543427wrc.49.1519683703340; Mon, 26 Feb 2018 14:21:43 -0800 (PST) Received: from dwarf (stargate.chelsio.com. [12.32.117.8]) by smtp.googlemail.com with ESMTPSA id l22sm12108176wre.52.2018.02.26.14.21.41 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 26 Feb 2018 14:21:42 -0800 (PST) Sender: Navdeep Parhar Message-ID: <1519683699.47932.5.camel@FreeBSD.org> Subject: Re: [HEADS UP] - OFED/RDMA stack update From: Navdeep Parhar To: Meny Yossefi , "'freebsd-infiniband@freebsd.org'" , "'FreeBSD-stable@FreeBSD.org'" , freebsd-arch Cc: freebsd-drivers Date: Mon, 26 Feb 2018 14:21:39 -0800 In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.24.2 FreeBSD GNOME Team Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Feb 2018 22:21:49 -0000 +freebsd-arch@ Hi Meny, Can you please post the KPI/KBI analysis that you generated to some public location and provide a link here? A straight MFC would be a major break of KPI/KBI in -STABLE and the options we're looking at are: a) Ignore the breakage and let downstream consumers deal with the fallout. This obviously isn't ideal in a -STABLE branch. b) Provide compat shims to at least preserve the KPI. One challenge is that the changes include functions with the same name but different signature/behavior. See, for example, ib_create_cq in Meny's list once he publishes it. c) Have two versions of the OFED interfaces in 11-STABLE and not break existing downstream consumers at all. I've reached out to users that I know of and know will be affected. If you use OFED and FreeBSD 11 this would be a good time to weigh in with your thoughts, ideas, concerns etc.. Regards, Navdeep On Mon, 2018-02-26 at 14:49 +0000, Meny Yossefi wrote: > Hi, > > We are currently working on MFC'ing the OFED/RDMA stack update > mentioned below to FreeBSD11-STABLE. > > We already have working version in 'projects/bsd_rdma_4_9_stable_11' > (pending iWARP updates) and currently working on ULP integration. > > Again, as always, for any concern/comments you might have, please > don't hesitate contacting us. > > freebsd-drivers@mellanox.com > > > Regards, > > Meny Yossefi, > Mellanox technologies > > > -----Original Message----- > From: Meny Yossefi > Sent: Monday, November 13, 2017 11:09 AM > To: 'freebsd-infiniband@freebsd.org' ; > 'freebsd-current@freebsd.org' > Cc: freebsd-drivers > Subject: [HEADS UP] - OFED/RDMA stack update > > Hi, > > This is to inform you that by end of this week we plan to merge the > OFED/RDMA stack update from the project - 'bsd_rdma_4_9' into 12- > CURRENT. > The update aligns the OFED common code and RDMA vendor drivers with > Linux v4.9. > > We are still working on final modifications and build testing it prior > to submission. > > > For any concern/comments you might have, please don't hesitate > contacting us. > > freebsd-drivers@mellanox.com > > > Regards, > > Meny Yossefi, > Mellanox technologies > _______________________________________________ > freebsd-infiniband@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-infiniband > To unsubscribe, send any mail to "freebsd-infiniband-unsubscribe@freeb > sd.org" From owner-freebsd-infiniband@freebsd.org Mon Feb 26 22:43:27 2018 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A0D82F29C9D; Mon, 26 Feb 2018 22:43:27 +0000 (UTC) (envelope-from kib@freebsd.org) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F18DB7086C; Mon, 26 Feb 2018 22:43:26 +0000 (UTC) (envelope-from kib@freebsd.org) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id w1QMhBXt073562 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 27 Feb 2018 00:43:14 +0200 (EET) (envelope-from kib@freebsd.org) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua w1QMhBXt073562 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id w1QMhBvk073561; Tue, 27 Feb 2018 00:43:11 +0200 (EET) (envelope-from kib@freebsd.org) X-Authentication-Warning: tom.home: kostik set sender to kib@freebsd.org using -f Date: Tue, 27 Feb 2018 00:43:11 +0200 From: Konstantin Belousov To: Navdeep Parhar Cc: Meny Yossefi , "'freebsd-infiniband@freebsd.org'" , "'FreeBSD-stable@FreeBSD.org'" , freebsd-arch , freebsd-drivers Subject: Re: [HEADS UP] - OFED/RDMA stack update Message-ID: <20180226224311.GT94212@kib.kiev.ua> References: <1519683699.47932.5.camel@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1519683699.47932.5.camel@FreeBSD.org> User-Agent: Mutt/1.9.3 (2018-01-21) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Feb 2018 22:43:27 -0000 On Mon, Feb 26, 2018 at 02:21:39PM -0800, Navdeep Parhar wrote: > +freebsd-arch@ > > Hi Meny, > > Can you please post the KPI/KBI analysis that you generated to some > public location and provide a link here? A straight MFC would be a > major break of KPI/KBI in -STABLE and the options we're looking at are: I put the report at https://kib.kiev.ua/kib/ibcore_11_to_11_merged_compat_report.html > > a) Ignore the breakage and let downstream consumers deal with the > fallout. This obviously isn't ideal in a -STABLE branch. > > b) Provide compat shims to at least preserve the KPI. One challenge is > that the changes include functions with the same name but different > signature/behavior. See, for example, ib_create_cq in Meny's list once > he publishes it. Project did handled similar issues already. One of the approaches is to renname the ib_create_cq with the new signature to ib_create_cq_n12 and check for (e.g.) _WANT_NEW_OFED symbol and to select one or another: #ifdef _WANT_NEW_OFED #define ib_create_cq(new args there) ib_create_cq_n21(new args there) #else #define ib_create_cq (ib_create_cq) #endif Then ULP that wants new KPI defines _WANT_NEW_OFED. > > c) Have two versions of the OFED interfaces in 11-STABLE and not break > existing downstream consumers at all. It is possible to make them loadable simultaneously as modules, but it is quite confusing to users, because Mellanox clearly wants mlx5_ib and mlx4_ib to work only with new OFED, while cxgbe would use old OFED ? Also, either we would need to mess with the ibcore.ko module name, or with version. I am not sure that our module handling is robust enough to make the version trick possible. > > I've reached out to users that I know of and know will be affected. > If you use OFED and FreeBSD 11 this would be a good time to weigh > in with your thoughts, ideas, concerns etc..