From owner-freebsd-infiniband@freebsd.org Sun Apr 24 10:39:00 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 63CE5B19B3A for ; Sun, 24 Apr 2016 10:39:00 +0000 (UTC) (envelope-from menyy@mellanox.com) Received: from emea01-am1-obe.outbound.protection.outlook.com (mail-am1on0061.outbound.protection.outlook.com [157.56.112.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DA01B136B for ; Sun, 24 Apr 2016 10:38:59 +0000 (UTC) (envelope-from menyy@mellanox.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:To:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=lmj0h7HVHxUaTgghlJoQHkACjgk6fchVBDy9t+q29qA=; b=D9X+qa+o/v6rnZla+WHpp9iM+x5gn4pOdqXPDnU4JA/8F1ODhho8S+vIwiA3Bsh5rrZ483cGfAecfs4jvP7ZSlwBvcTdcX/pRmWfir9xjfEf520G45G9CFZx+V38X0eiVHH50nxFJYx5lZ+TSfklElqUaZGgiD4h8spXLSwqOyk= Received: from DB5PR05MB1765.eurprd05.prod.outlook.com (10.165.7.23) by DB5PR05MB1894.eurprd05.prod.outlook.com (10.166.173.139) with Microsoft SMTP Server (TLS) id 15.1.466.19; Sun, 24 Apr 2016 05:04:34 +0000 Received: from DB5PR05MB1765.eurprd05.prod.outlook.com ([10.165.7.23]) by DB5PR05MB1765.eurprd05.prod.outlook.com ([10.165.7.23]) with mapi id 15.01.0466.024; Sun, 24 Apr 2016 05:04:34 +0000 From: Meny Yossefi To: Mark Bloch , Yuval Bason , "'freebsd-infiniband@freebsd.org'" CC: Hans Petter Selasky Subject: RE: Table of supported IB hw per FreeBSD version? Thread-Topic: Table of supported IB hw per FreeBSD version? Thread-Index: AQHRm10OTwqpfw5q1kWrAxF6iF/ZBZ+UsLMAgAANAwCAAB+NgIAAAIeAgAAAE8uAATLewIAAH5oAgAAAGN6AAmVr4A== Date: Sun, 24 Apr 2016 05:04:34 +0000 Message-ID: References: <4C75517F-4EA4-4B42-833D-F43094E9DAD3@postgresql.org> <677214B0-6A92-42AA-A7B3-D125CB826761@jnielsen.net> , <7BB80C17-897C-40B2-9731-A750A9EDD290@postgresql.org> <3185e0b5eb704dfc99c65aaed07eb215@AMSPR05MB0580.eurprd05.prod.outlook.com> , In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: mellanox.com; dkim=none (message not signed) header.d=none;mellanox.com; dmarc=none action=none header.from=mellanox.com; x-originating-ip: [193.47.165.251] x-ms-office365-filtering-correlation-id: b3d86f71-816e-4508-6f7b-08d36bfdedbb x-microsoft-exchange-diagnostics: 1; DB5PR05MB1894; 5:U5nAqOQzUu6kgYEK5OsItk9ffSbsDN2xofBDWy+R9dK1w5dp1K9eudC+4lXSyqknmheTz2E16kopQBbK8hKxZe6nAO244k9VNr3iuQhy8RyVq6/A3E8Yc57XwFRMRVm0t64kMwbOZ7H7emQwbaBgtX1o6BKlprQBQmGOJJP/pLqUljaE0+lnNo1uxSQLVooI; 24:5iQ5v2x5QUZpTHrIhtJD7qYvY24IoyFInLv44Zlf5rJbYCdBclkkx6bpESb4Ju8Ja73b45OpiGWz3ZBYwc8nYFZmVN2rkW5KKrpKfn+kWkQ=; 7:hI9JJLDIz20INHb6jCRs0B6rw/1Dr0jLafjbMMpMEnjgYfnkgkJoemYwEZiHK+99coNg5gyAkM6B9s4qBVCn3T1mJcKNlWZmJJcWGKj87oxkyLJFMD3CDXda9ZaF9jrhN0XXV6TfHbUqUZNgXa8uuMRhebIZIcYSIZb02fP1RoQS2bkbVUPR7hgxxhJoX1SS7HNprNE8IAxgCTl84uPbWw== x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:DB5PR05MB1894; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(9101521062)(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046)(6055026); SRVR:DB5PR05MB1894; BCL:0; PCL:0; RULEID:; SRVR:DB5PR05MB1894; x-forefront-prvs: 09222B39F5 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6009001)(24454002)(377454003)(10400500002)(33656002)(92566002)(3280700002)(3900700001)(50986999)(5001770100001)(19580395003)(19580405001)(5002640100001)(5004730100002)(5008740100001)(4001430100002)(15975445007)(11100500001)(81166005)(76576001)(9686002)(66066001)(54356999)(76176999)(2900100001)(2950100001)(3660700001)(450100001)(189998001)(77096005)(102836003)(3846002)(6116002)(586003)(2906002)(87936001)(1220700001)(1096002)(5003600100002)(106116001)(74316001)(93886004)(107886002)(4326007)(122556002)(86362001)(491001); DIR:OUT; SFP:1101; SCL:1; SRVR:DB5PR05MB1894; H:DB5PR05MB1765.eurprd05.prod.outlook.com; FPR:; SPF:None; MLV:sfv; LANG:en; Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Apr 2016 05:04:34.7660 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB5PR05MB1894 X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Apr 2016 10:39:00 -0000 Hi,=20 As for 9.x, no problem stating them as "No".=20 -Meny -----Original Message----- From: freebsd-commits-tracker=20 Sent: Friday, April 22, 2016 7:25 PM To: Mark Bloch ; Meny Yossefi ; Yuv= al Bason Subject: FW: Table of supported IB hw per FreeBSD version? ________________________________________ From: owner-freebsd-infiniband@freebsd.orgOn Behalf OfJustin Clift Sent: Friday, April 22, 2016 4:25:06 PM (UTC) Monrovia, Reykjavik To: Meny Yossefi Cc: freebsd-infiniband@freebsd.org; Yuval Bason; Hans Petter Selasky Subject: Re: Table of supported IB hw per FreeBSD version? On 22 Apr 2016, at 15:52, Meny Yossefi wrote: > Here's an updated list: Thanks Meny, that's excellent. :) Transcribed it to the table here: https://wiki.freebsd.org/InfiniBand For the "N/A" entries - newer hardware on FreeBSD 9.x - would it be ok to j= ust have them as "No"? Am thinking it would help keep things mentally simp= le for people to process that way. eg "Yes", "No", "Not Yet" are all straight forward answers. "N/A" instead can raise questions, as people have to try and figure what "N/A" means in this context. Workable? :) Regards and best wishes, Justin Clift -- "My grandfather once told me that there are two kinds of people: those who = work and those who take the credit. He told me to try to be in the first gr= oup; there was less competition there." - Indira Gandhi _______________________________________________ freebsd-infiniband@freebsd.org mailing list https://lists.freebsd.org/mailm= an/listinfo/freebsd-infiniband To unsubscribe, send any mail to "freebsd-infiniband-unsubscribe@freebsd.or= g" From owner-freebsd-infiniband@freebsd.org Sun Apr 24 11:11:46 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C91F9B1B314 for ; Sun, 24 Apr 2016 11:11:46 +0000 (UTC) (envelope-from justin@postgresql.org) Received: from meldrar.postgresql.org (meldrar.postgresql.org [IPv6:2a02:c0:301:0:ffff::31]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "*.postgresql.org", Issuer "Gandi Standard SSL CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8939A126B for ; Sun, 24 Apr 2016 11:11:46 +0000 (UTC) (envelope-from justin@postgresql.org) Received: from 82-69-92-196.dsl.in-addr.zen.co.uk ([82.69.92.196] helo=[172.16.1.14]) by meldrar.postgresql.org with esmtpsa (TLS1.0:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1auHwx-00032a-4h; Sun, 24 Apr 2016 11:11:43 +0000 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Table of supported IB hw per FreeBSD version? From: Justin Clift In-Reply-To: Date: Sun, 24 Apr 2016 12:11:37 +0100 Cc: Mark Bloch , Yuval Bason , Hans Petter Selasky , freebsd-infiniband@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <2E355FDF-7D57-494B-8B63-554C0759C1CD@postgresql.org> References: <4C75517F-4EA4-4B42-833D-F43094E9DAD3@postgresql.org> <677214B0-6A92-42AA-A7B3-D125CB826761@jnielsen.net> , <7BB80C17-897C-40B2-9731-A750A9EDD290@postgresql.org> <3185e0b5eb704dfc99c65aaed07eb215@AMSPR05MB0580.eurprd05.prod.outlook.com> , To: Meny Yossefi X-Mailer: Apple Mail (2.1878.6) X-Pg-Spam-Score: -2.9 (--) X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Apr 2016 11:11:46 -0000 Thanks Meny, done. :) + Justin On 24 Apr 2016, at 06:04, Meny Yossefi wrote: > Hi,=20 >=20 > As for 9.x, no problem stating them as "No".=20 >=20 > -Meny >=20 >=20 > -----Original Message----- > From: freebsd-commits-tracker=20 > Sent: Friday, April 22, 2016 7:25 PM > To: Mark Bloch ; Meny Yossefi = ; Yuval Bason > Subject: FW: Table of supported IB hw per FreeBSD version? >=20 >=20 >=20 >=20 > ________________________________________ > From: owner-freebsd-infiniband@freebsd.orgOn Behalf OfJustin Clift > Sent: Friday, April 22, 2016 4:25:06 PM (UTC) Monrovia, Reykjavik > To: Meny Yossefi > Cc: freebsd-infiniband@freebsd.org; Yuval Bason; Hans Petter Selasky > Subject: Re: Table of supported IB hw per FreeBSD version? >=20 > On 22 Apr 2016, at 15:52, Meny Yossefi wrote: > >> Here's an updated list: >=20 > Thanks Meny, that's excellent. :) >=20 > Transcribed it to the table here: >=20 > https://wiki.freebsd.org/InfiniBand >=20 > For the "N/A" entries - newer hardware on FreeBSD 9.x - would it be ok = to just have them as "No"? Am thinking it would help keep things = mentally simple for people to process that way. >=20 > eg "Yes", "No", "Not Yet" are all straight forward answers. "N/A" > instead can raise questions, as people have to try and figure what > "N/A" means in this context. >=20 > Workable? :) >=20 > Regards and best wishes, >=20 > Justin Clift >=20 > -- > "My grandfather once told me that there are two kinds of people: those = who work and those who take the credit. He told me to try to be in the = first group; there was less competition there." > - Indira Gandhi >=20 > _______________________________________________ > freebsd-infiniband@freebsd.org mailing list = https://lists.freebsd.org/mailman/listinfo/freebsd-infiniband > To unsubscribe, send any mail to = "freebsd-infiniband-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-infiniband@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-infiniband > To unsubscribe, send any mail to = "freebsd-infiniband-unsubscribe@freebsd.org" -- "My grandfather once told me that there are two kinds of people: those who work and those who take the credit. He told me to try to be in the first group; there was less competition there." - Indira Gandhi From owner-freebsd-infiniband@freebsd.org Sun Apr 24 11:34:07 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 085BAB1B650 for ; Sun, 24 Apr 2016 11:34:07 +0000 (UTC) (envelope-from justin@postgresql.org) Received: from meldrar.postgresql.org (meldrar.postgresql.org [IPv6:2a02:c0:301:0:ffff::31]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "*.postgresql.org", Issuer "Gandi Standard SSL CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C419F1B76 for ; Sun, 24 Apr 2016 11:34:06 +0000 (UTC) (envelope-from justin@postgresql.org) Received: from 82-69-92-196.dsl.in-addr.zen.co.uk ([82.69.92.196] helo=[172.16.1.14]) by meldrar.postgresql.org with esmtpsa (TLS1.0:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1auIIc-0003Up-9f for freebsd-infiniband@freebsd.org; Sun, 24 Apr 2016 11:34:05 +0000 From: Justin Clift Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: Status items on the Infiniband wiki page? Message-Id: <80192BB9-7084-4D3E-8029-6E6D921752F8@postgresql.org> Date: Sun, 24 Apr 2016 12:34:02 +0100 To: freebsd-infiniband@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) X-Mailer: Apple Mail (2.1878.6) X-Pg-Spam-Score: -2.9 (--) X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Apr 2016 11:34:07 -0000 Hi all, There's a small "Status" section on the Infiniband wiki page: It has these items on there, but they're not all that informative: Short term items * Upgrade to 1.5.x (Isilon). Delivery date: TBD. Long term items * Upgrade to 3.x (Isilon). Can't be done immediately because of refactoring done in the kernel.org branch and with the userland utilities, and the fact that things are still in a large degree of flux. Are those items still correct for the "Status" section? If so, can someone please explain what they mean? Reading through the -infiniband mailing list didn't really help shed light on them. ;) Next thought, we should probably add information about the "known good" vs "needs work" parts of the IB stack. For example, IPoIB and IPoIB connected mode seem like they could be better. Hans is improving the IPoIB code (in 11.x?) which is fantastic. :) We can still set people's expectations correctly for the users and potential users of earlier releases though. Are there any other parts of the stack which stand out as being excellent, or not that great? Regards and best wishes, Justin Clift -- "My grandfather once told me that there are two kinds of people: those who work and those who take the credit. He told me to try to be in the first group; there was less competition there." - Indira Gandhi From owner-freebsd-infiniband@freebsd.org Sun Apr 24 11:35:18 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 452EEB1B70B for ; Sun, 24 Apr 2016 11:35:18 +0000 (UTC) (envelope-from justin@postgresql.org) Received: from meldrar.postgresql.org (meldrar.postgresql.org [IPv6:2a02:c0:301:0:ffff::31]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "*.postgresql.org", Issuer "Gandi Standard SSL CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0B2A01BBE for ; Sun, 24 Apr 2016 11:35:18 +0000 (UTC) (envelope-from justin@postgresql.org) Received: from 82-69-92-196.dsl.in-addr.zen.co.uk ([82.69.92.196] helo=[172.16.1.14]) by meldrar.postgresql.org with esmtpsa (TLS1.0:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1auIJl-0003Vn-MB for freebsd-infiniband@freebsd.org; Sun, 24 Apr 2016 11:35:16 +0000 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Status items on the Infiniband wiki page? From: Justin Clift In-Reply-To: <80192BB9-7084-4D3E-8029-6E6D921752F8@postgresql.org> Date: Sun, 24 Apr 2016 12:35:14 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <98D5D075-3BB6-459B-949D-EC63CA2A2B39@postgresql.org> References: <80192BB9-7084-4D3E-8029-6E6D921752F8@postgresql.org> To: freebsd-infiniband@freebsd.org X-Mailer: Apple Mail (2.1878.6) X-Pg-Spam-Score: -2.9 (--) X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Apr 2016 11:35:18 -0000 Gah, forgot the link again. ;) https://wiki.freebsd.org/InfiniBand On 24 Apr 2016, at 12:34, Justin Clift wrote: > Hi all, >=20 > There's a small "Status" section on the Infiniband wiki page: >=20 >=20 > It has these items on there, but they're not all that informative: >=20 > Short term items > * Upgrade to 1.5.x (Isilon). Delivery date: TBD. >=20 > Long term items > * Upgrade to 3.x (Isilon). Can't be done immediately because of > refactoring done in the kernel.org branch and with the userland > utilities, and the fact that things are still in a large degree > of flux. >=20 > Are those items still correct for the "Status" section? If so, can > someone please explain what they mean? Reading through the = -infiniband > mailing list didn't really help shed light on them. ;) >=20 > Next thought, we should probably add information about the "known = good" > vs "needs work" parts of the IB stack. For example, IPoIB and IPoIB > connected mode seem like they could be better. >=20 > Hans is improving the IPoIB code (in 11.x?) which is fantastic. :) >=20 > We can still set people's expectations correctly for the users and > potential users of earlier releases though. Are there any other parts > of the stack which stand out as being excellent, or not that great? >=20 > Regards and best wishes, >=20 > Justin Clift >=20 > -- > "My grandfather once told me that there are two kinds of people: those > who work and those who take the credit. He told me to try to be in the > first group; there was less competition there." > - Indira Gandhi >=20 > _______________________________________________ > freebsd-infiniband@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-infiniband > To unsubscribe, send any mail to = "freebsd-infiniband-unsubscribe@freebsd.org" -- "My grandfather once told me that there are two kinds of people: those who work and those who take the credit. He told me to try to be in the first group; there was less competition there." - Indira Gandhi From owner-freebsd-infiniband@freebsd.org Sun Apr 24 13:14:25 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8D9A0B19483 for ; Sun, 24 Apr 2016 13:14:25 +0000 (UTC) (envelope-from bacon4000@gmail.com) Received: from mail-io0-x22f.google.com (mail-io0-x22f.google.com [IPv6:2607:f8b0:4001:c06::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 57A381E2D for ; Sun, 24 Apr 2016 13:14:25 +0000 (UTC) (envelope-from bacon4000@gmail.com) Received: by mail-io0-x22f.google.com with SMTP id u185so160592016iod.3 for ; Sun, 24 Apr 2016 06:14:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=WPNat9iws/78ddI0kosXaBi1JTvbUxYk8wjBlySCeiA=; b=vwbr+aciSGjEHAoiUnWUH3zsG7NsGb9Wkhxp7wvbWqYGAKTLh3BzROSVUgyFOhAOks Li7VuD40bd+6cVvu79emdPn0wGYXyZfF3xI1sDxcmaSTaO0XxE/CZMZvugYd0gDWylaK uSrSLxkBVSGFdlpcrN2GO1/f7g8I4ZOxt4Zc4D8xNw3I9WI5FmTHnr3MNGhpBJwTppyL LglkkdDTzUmGDLrtJFlJLj6R5wkZWzAtJRongRJm6dpUHykhPr35YTi342HnrG00YKjW HpALhcWR9OSWZWTAM6X969+M6v7XCkJ2XSMhqla2fcvRjhnn7Xd1GCRXnBLNrHFqr1gQ t+cA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=WPNat9iws/78ddI0kosXaBi1JTvbUxYk8wjBlySCeiA=; b=CtmPgbPunyfckmBZ1Ny0kBYmzoizVc2Jv26sKdXP46hL5CyTIR1djWG2KHobPX7xE4 NPY3PqeA9k6zvZxyg/zfl5X6BQNd9IpW1lZO75X2kuAWQAToWuSouih3okzuZSjIzg0K q4OlZ2qU5LNU+XOEeo8xGsO3ym3fJL4FXyjLD1EUCK7ZqY2MMrcI04fqjXbAcARNiw56 WkhFPdNUCyGicj+tt/AJMAdFjeUcUTKMwxcWBU0VzZ7o6U70uyZ7yugHkXebhM1V4Is8 TbAKedgNXXdGpS7gp9LkW0GpZDbr9tJ/fg1YRte9s9CLuZWGQq6+EoX38NQt9CmeDHla VihQ== X-Gm-Message-State: AOPr4FV02g5iWV8M9zY4e4zgbtMWH5P/egIt+RRzgIfkd1lysfLGUEO+hz1QcKtLk9xlcA== X-Received: by 10.107.58.197 with SMTP id h188mr31378138ioa.125.1461503664672; Sun, 24 Apr 2016 06:14:24 -0700 (PDT) Received: from imacbsd.acadix.biz ([64.83.177.9]) by smtp.gmail.com with ESMTPSA id s8sm6632353igg.9.2016.04.24.06.14.23 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 24 Apr 2016 06:14:24 -0700 (PDT) Subject: Re: Kernel modules To: Justin Clift References: <57194802.8090403@gmail.com> <643721CB-BB8D-43AA-933B-C8E10E5CDBD5@postgresql.org> Cc: freebsd-infiniband@freebsd.org From: Jason Bacon Message-ID: <571CC6AC.3040008@gmail.com> Date: Sun, 24 Apr 2016 08:14:20 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.7.1 MIME-Version: 1.0 In-Reply-To: <643721CB-BB8D-43AA-933B-C8E10E5CDBD5@postgresql.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Apr 2016 13:14:25 -0000 I found the problem and have successfully configured my IB card using only kernel modules and buildworld. I documented the process on the wiki. Regards, Jason On 04/21/16 17:56, Justin Clift wrote: > On 21 Apr 2016, at 22:37, Jason Bacon wrote: > >> It runs cleanly on my current 10.2-RELEASE system, but my ConnectX-3 card is not recognized. > Just to clarify, the system is showing the ConnectX-3 card is on the > PCI bus (eg through pciconf -lvV), but the driver isn't attaching to it? > > + Justin > > -- > "My grandfather once told me that there are two kinds of people: those > who work and those who take the credit. He told me to try to be in the > first group; there was less competition there." > - Indira Gandhi > -- All wars are civil wars, because all men are brothers ... Each one owes infinitely more to the human race than to the particular country in which he was born. -- Francois Fenelon From owner-freebsd-infiniband@freebsd.org Sun Apr 24 14:27:59 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 97CEDB1A499 for ; Sun, 24 Apr 2016 14:27:59 +0000 (UTC) (envelope-from justin@postgresql.org) Received: from meldrar.postgresql.org (meldrar.postgresql.org [IPv6:2a02:c0:301:0:ffff::31]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "*.postgresql.org", Issuer "Gandi Standard SSL CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5E29D11EC for ; Sun, 24 Apr 2016 14:27:59 +0000 (UTC) (envelope-from justin@postgresql.org) Received: from 82-69-92-196.dsl.in-addr.zen.co.uk ([82.69.92.196] helo=[172.16.1.14]) by meldrar.postgresql.org with esmtpsa (TLS1.0:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1auL0s-0007VK-KS; Sun, 24 Apr 2016 14:27:57 +0000 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Kernel modules From: Justin Clift In-Reply-To: <571CC6AC.3040008@gmail.com> Date: Sun, 24 Apr 2016 15:27:54 +0100 Cc: freebsd-infiniband@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <2740FF7B-D159-4E1C-8CF6-DE9D3FF08D4D@postgresql.org> References: <57194802.8090403@gmail.com> <643721CB-BB8D-43AA-933B-C8E10E5CDBD5@postgresql.org> <571CC6AC.3040008@gmail.com> To: Jason Bacon X-Mailer: Apple Mail (2.1878.6) X-Pg-Spam-Score: -2.9 (--) X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Apr 2016 14:27:59 -0000 On 24 Apr 2016, at 14:14, Jason Bacon wrote: > I found the problem and have successfully configured my IB card using = only kernel modules and buildworld. >=20 > I documented the process on the wiki. Awesome, that definitely helps. :) + Justin -- "My grandfather once told me that there are two kinds of people: those who work and those who take the credit. He told me to try to be in the first group; there was less competition there." - Indira Gandhi From owner-freebsd-infiniband@freebsd.org Mon Apr 25 14:03:52 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5C7E0B1B279 for ; Mon, 25 Apr 2016 14:03:52 +0000 (UTC) (envelope-from justin@postgresql.org) Received: from meldrar.postgresql.org (meldrar.postgresql.org [IPv6:2a02:c0:301:0:ffff::31]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "*.postgresql.org", Issuer "Gandi Standard SSL CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2015D1FF2 for ; Mon, 25 Apr 2016 14:03:51 +0000 (UTC) (envelope-from justin@postgresql.org) Received: from 82-69-92-196.dsl.in-addr.zen.co.uk ([82.69.92.196] helo=[172.16.1.14]) by meldrar.postgresql.org with esmtpsa (TLS1.0:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1auh73-0007I2-OY for freebsd-infiniband@freebsd.org; Mon, 25 Apr 2016 14:03:48 +0000 From: Justin Clift Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: Initial ISO's ready for testing FreeNAS 9.10 with Mellanox cards Message-Id: <65B3A04E-53D0-49CB-8DE0-FDE0D8F66B26@postgresql.org> Date: Mon, 25 Apr 2016 15:03:45 +0100 To: freebsd-infiniband@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) X-Mailer: Apple Mail (2.1878.6) X-Pg-Spam-Score: -2.9 (--) X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Apr 2016 14:03:52 -0000 This is semi-off-topic, but hoping it's closely enough related that I'll be forgiven. :) If anyone has time/interest to try out FreeNAS 9.10 (based on FreeBSD 10.3) - with Mellanox Infiniband card drivers baked in, there are two ISO's here for testing: https://github.com/justinclift/freenas-infiniband=E2=80=8B One ISO is for native Infiniband mode (forcing IPoIB mode for the = cards): = https://github.com/justinclift/freenas-infiniband/releases/tag/FreeNAS-IB-= 9.10-alpha1 The other ISO is for native Ethernet mode: = https://github.com/justinclift/freenas-infiniband/releases/tag/FreeNAS-IB-= EN-9.10-alpha1v10 InfiniHost through to ConnectX-3 cards should all work for the IB mode one. ConnectX through ConnectX-3 should work for the Ethernet mode one. And yeah, it would be better to have just one ISO. There is a weird kernel panic bug I'm trying to track down first (triggered when switched modes), that needs fixing before that can happen. All feedback is hugely encouraged and welcome. :) Regards and best wishes, Justin Clift -- "My grandfather once told me that there are two kinds of people: those who work and those who take the credit. He told me to try to be in the first group; there was less competition there." - Indira Gandhi From owner-freebsd-infiniband@freebsd.org Mon Apr 25 14:37:28 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1F38DB1B9B9 for ; Mon, 25 Apr 2016 14:37:28 +0000 (UTC) (envelope-from bacon4000@gmail.com) Received: from mail-io0-x232.google.com (mail-io0-x232.google.com [IPv6:2607:f8b0:4001:c06::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DA07D1DBF for ; Mon, 25 Apr 2016 14:37:27 +0000 (UTC) (envelope-from bacon4000@gmail.com) Received: by mail-io0-x232.google.com with SMTP id f89so151104309ioi.0 for ; Mon, 25 Apr 2016 07:37:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-transfer-encoding; bh=AJfC1ROw/A4c90dBwEciaVMYOu29cRS87sDe3KIOgS4=; b=uUHFQkBGIxVn9p/ut9VBa3TLulRzTB04N69XVhfDsTFb3ArIUSDjnM/C2muZyORjwm GVesRtDaLPt4UGKHgTHMTHSaFvVCSAfOj5m/ntM0Awp6APOfzzYZX3PsHwmFoELGMWNr /PmyehXdpyeE65MOap3EiwjiRXgD+uTyi7jk0F6N0tcVRDF04TW6GUXQ2WKsqmGl8JRL IoRU9i3czlOwbIeEZCuMftWIAELD2NdTCKp29XdoPe5HxvCKnqSwuum7SXdMfVlWmAk8 HEK/vcJHW7yKVYw4zmodGCWP1zCGwk/S3yYHpUh4bJNOUT7dubYY5bzHLJCyz84T/lWx EXYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=AJfC1ROw/A4c90dBwEciaVMYOu29cRS87sDe3KIOgS4=; b=CRII/nWHfS/eY2+MBrS1w1SZaGRkkGiGJsVKU1l7FPwmNYIdeBqJMm0neHU2IqaW7j F5akYjqa1DKtN1u8MMy8byz0sAJOc1SL28J4Et1wxU75l2e/Tl2qDa081Tob0kc/FIeB iRgmHmBDxKbU1Nw+jXdrpa//BIrOLtbUa4lcayaTZ1C5FFYi8sKhPF951ZlzC78os8Na HBlT6ZG5dCLEwWUaWBzoQHJ7vexIeOjPlsDE0NmUEPDXq1yiTxidb7mDNDcl6LwU786v VjnCC0P/1riukcWup3yPxD21pN8e8iu8mwExX0I8frZlOL4HaAEh0ER6KUfY90irqvew ff7g== X-Gm-Message-State: AOPr4FVv8m1FvYv+7YjK4BiKGJM2774GDq5WpBVL+116VwL6t5YgzImSTrTkNRN2VHdXTA== X-Received: by 10.107.161.68 with SMTP id k65mr44170362ioe.110.1461595047346; Mon, 25 Apr 2016 07:37:27 -0700 (PDT) Received: from imacbsd.acadix.biz ([64.83.177.9]) by smtp.gmail.com with ESMTPSA id f71sm11568330iof.1.2016.04.25.07.37.26 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 25 Apr 2016 07:37:26 -0700 (PDT) Subject: Re: Initial ISO's ready for testing FreeNAS 9.10 with Mellanox cards To: Justin Clift , freebsd-infiniband@freebsd.org References: <65B3A04E-53D0-49CB-8DE0-FDE0D8F66B26@postgresql.org> From: Jason Bacon Message-ID: <571E2BA5.5080500@gmail.com> Date: Mon, 25 Apr 2016 09:37:25 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.7.1 MIME-Version: 1.0 In-Reply-To: <65B3A04E-53D0-49CB-8DE0-FDE0D8F66B26@postgresql.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Apr 2016 14:37:28 -0000 Justin, This is *exactly* what I'm interested in doing right now. In fact, I had been exploring how to do a FreeNAS build with IB support. Thanks a million for providing the ISOs. I can test with Infinihost and ConnectX-3. Jason On 04/25/16 09:03, Justin Clift wrote: > This is semi-off-topic, but hoping it's closely enough related that I'll > be forgiven. :) > > If anyone has time/interest to try out FreeNAS 9.10 (based on FreeBSD > 10.3) - with Mellanox Infiniband card drivers baked in, there are two > ISO's here for testing: > > https://github.com/justinclift/freenas-infiniband​ > > One ISO is for native Infiniband mode (forcing IPoIB mode for the cards): > > https://github.com/justinclift/freenas-infiniband/releases/tag/FreeNAS-IB-9.10-alpha1 > > The other ISO is for native Ethernet mode: > > https://github.com/justinclift/freenas-infiniband/releases/tag/FreeNAS-IB-EN-9.10-alpha1v10 > > InfiniHost through to ConnectX-3 cards should all work for the IB mode > one. ConnectX through ConnectX-3 should work for the Ethernet mode one. > > And yeah, it would be better to have just one ISO. There is a weird > kernel panic bug I'm trying to track down first (triggered when switched > modes), that needs fixing before that can happen. > > All feedback is hugely encouraged and welcome. :) > > Regards and best wishes, > > Justin Clift > > -- > "My grandfather once told me that there are two kinds of people: those > who work and those who take the credit. He told me to try to be in the > first group; there was less competition there." > - Indira Gandhi > > _______________________________________________ > freebsd-infiniband@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-infiniband > To unsubscribe, send any mail to "freebsd-infiniband-unsubscribe@freebsd.org" -- All wars are civil wars, because all men are brothers ... Each one owes infinitely more to the human race than to the particular country in which he was born. -- Francois Fenelon From owner-freebsd-infiniband@freebsd.org Mon Apr 25 14:46:43 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9568AB1BD32 for ; Mon, 25 Apr 2016 14:46:43 +0000 (UTC) (envelope-from justin@postgresql.org) Received: from meldrar.postgresql.org (meldrar.postgresql.org [IPv6:2a02:c0:301:0:ffff::31]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "*.postgresql.org", Issuer "Gandi Standard SSL CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5B79D1498 for ; Mon, 25 Apr 2016 14:46:43 +0000 (UTC) (envelope-from justin@postgresql.org) Received: from meldrar.postgresql.org ([87.238.57.231] helo=webmail.postgresql.org) by meldrar.postgresql.org with esmtpsa (TLS1.0:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.84_2) (envelope-from ) id 1auhmY-0008Ne-K3; Mon, 25 Apr 2016 14:46:41 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Mon, 25 Apr 2016 15:46:37 +0100 From: Justin Clift To: Jason Bacon Cc: freebsd-infiniband@freebsd.org Subject: Re: Initial ISO's ready for testing FreeNAS 9.10 with Mellanox cards In-Reply-To: <571E2BA5.5080500@gmail.com> References: <65B3A04E-53D0-49CB-8DE0-FDE0D8F66B26@postgresql.org> <571E2BA5.5080500@gmail.com> Message-ID: <157d89e5f246d88cb08fa43f2fbf690e@postgresql.org> X-Sender: justin@postgresql.org User-Agent: Roundcube Webmail/1.1.4 X-Pg-Spam-Score: -3.9 (---) X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Apr 2016 14:46:43 -0000 Awesome, please do. :) Noticed your post on the FreeNAS forum about this as well, so I've posted info there too. The post is in moderation though, so hopefully someone will approve it and others will test it too. :) As a side note, I've been able to do basic interoperability testing of the Ethernet mode ISO FreeNAS build with a Windows client (also running ConnectX card). Trying the same thing in IPoIB mode isn't working though. Seems like driver problems on the Windows side. :( Currently seeking assistance with that through the Mellanox Community forum (don't laugh... I'm not a Windows guy! :>). It's probably just something simple. Hopefully. ;) + Justin On 2016-04-25 15:37, Jason Bacon wrote: > Justin, > > This is *exactly* what I'm interested in doing right now. In fact, I > had been exploring how to do a FreeNAS build with IB support. Thanks > a million for providing the ISOs. > > I can test with Infinihost and ConnectX-3. > > Jason > > On 04/25/16 09:03, Justin Clift wrote: >> This is semi-off-topic, but hoping it's closely enough related that >> I'll >> be forgiven. :) >> >> If anyone has time/interest to try out FreeNAS 9.10 (based on FreeBSD >> 10.3) - with Mellanox Infiniband card drivers baked in, there are two >> ISO's here for testing: >> >> https://github.com/justinclift/freenas-infiniband​ >> >> One ISO is for native Infiniband mode (forcing IPoIB mode for the >> cards): >> >> >> https://github.com/justinclift/freenas-infiniband/releases/tag/FreeNAS-IB-9.10-alpha1 >> >> The other ISO is for native Ethernet mode: >> >> >> https://github.com/justinclift/freenas-infiniband/releases/tag/FreeNAS-IB-EN-9.10-alpha1v10 >> >> InfiniHost through to ConnectX-3 cards should all work for the IB mode >> one. ConnectX through ConnectX-3 should work for the Ethernet mode >> one. >> >> And yeah, it would be better to have just one ISO. There is a weird >> kernel panic bug I'm trying to track down first (triggered when >> switched >> modes), that needs fixing before that can happen. >> >> All feedback is hugely encouraged and welcome. :) >> >> Regards and best wishes, >> >> Justin Clift >> >> -- >> "My grandfather once told me that there are two kinds of people: those >> who work and those who take the credit. He told me to try to be in the >> first group; there was less competition there." >> - Indira Gandhi >> >> _______________________________________________ >> freebsd-infiniband@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-infiniband >> To unsubscribe, send any mail to >> "freebsd-infiniband-unsubscribe@freebsd.org" From owner-freebsd-infiniband@freebsd.org Mon Apr 25 21:55:50 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3E357B1C761 for ; Mon, 25 Apr 2016 21:55:50 +0000 (UTC) (envelope-from bacon4000@gmail.com) Received: from mail-io0-x22e.google.com (mail-io0-x22e.google.com [IPv6:2607:f8b0:4001:c06::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 098901941 for ; Mon, 25 Apr 2016 21:55:50 +0000 (UTC) (envelope-from bacon4000@gmail.com) Received: by mail-io0-x22e.google.com with SMTP id f89so162892466ioi.0 for ; Mon, 25 Apr 2016 14:55:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=to:from:subject:message-id:date:user-agent:mime-version :content-transfer-encoding; bh=/Y0D2wvwu6NROl8vZ5y2oHkv+h1AhAcWersDFE7vTDE=; b=olRM8FAWIRbFanrFN1+u4A7uDxTc3/AZY6JiivxxXh9H+wLzUZv0P7ZuabWLo4m6B9 GBCi4R/zJQssSPNVpBI4WD5x2YBG2NUN0kFhDdI3OLjMC/Zv2xHW0bXfGq93ErNx9IaO o3iAAhrQeiCbn67mjgypL4kE9ow/gSGSYXQBKQRk8sBp9DINkLPZd6TvErEkyzeVYH3i mV0D4WHtazRFAsycDzO6GOMJu2I3/LeGpx9XzQN7Ia/SPdp6RCr35Qn/tW/rlQynAOuY MRCYvcOKEbjRRP4DZCg7WIJuNal/cu0tDLDUVqfqatUbYLyWwpgAwfJ05XtXWEjWWEfa 2g6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:to:from:subject:message-id:date:user-agent :mime-version:content-transfer-encoding; bh=/Y0D2wvwu6NROl8vZ5y2oHkv+h1AhAcWersDFE7vTDE=; b=kEMwrHXaD8wkSlHxVjKx84tG/kWQZe7vuNqQE2ekljAgbeqp9PwBJp4foYXS05Ef29 sKxb5RY3Ca0dLBa5jpEmpvsopq0fXOMyQlT4jCF5WttpCQ5NFk1CPMZSpR4K3rzFgu7Y NwEB8mG7PoLDjCzBTfupEylbj+V24cGkCHM5LYH580x8o1JhR05SR0MwiXcpDUP66Ox3 EClsjeTPxgsC1Fp/RKcKRzcWcLnSb5ZcVLsqnQtoWmkKhnAPiTei7H36hgZczo1Ua1KS tROOZXOg2+mXQ9ti7kR0RFsoJGp5F3YLc5ofksWAHo2IJXr1ZNRKSLsdnR4Ov4yKWVCw tZYw== X-Gm-Message-State: AOPr4FUsxTT6fLmwqwJuHnP79sf6JL2sIiAhhmL6qojEajRFVICSVaogfqzNUOyf0Ok0Cw== X-Received: by 10.107.10.37 with SMTP id u37mr14875148ioi.92.1461621349295; Mon, 25 Apr 2016 14:55:49 -0700 (PDT) Received: from imacbsd.acadix.biz ([64.83.177.9]) by smtp.gmail.com with ESMTPSA id n75sm12485941ioe.9.2016.04.25.14.55.47 for (version=TLSv1/SSLv3 cipher=OTHER); Mon, 25 Apr 2016 14:55:48 -0700 (PDT) To: "freebsd-infiniband@freebsd.org" From: Jason Bacon Subject: Userland without buildworld? Message-ID: <571E9261.1000508@gmail.com> Date: Mon, 25 Apr 2016 16:55:45 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.7.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Apr 2016 21:55:50 -0000 I explored /usr/src a bit to see where WITH_OFED actually comes into play. For the most part, it appears to be just adding the following to the build tree: /usr/src/contrib/ofed /usr/src/sys/ofed Outside the ofed directories and the kernel, it doesn't seem to do a lot, as I would expect. /usr/src/etc/rc.d/Makefile optionally builds opensm. /usr/src/usr.bin/netstat/Makefile adds --DSDP to CFLAGS. It would seem to me that it should be possible to build OFED userland support without wasting time building the > 99% of the code that has nothing to do with it. Does anyone know of a proven method to build and install the OFED userland without doing an entire buildworld? This would go a long way toward easing deployment of the IB stack en masse. I suspect that there would be stiff resistance to the idea of including IB support in the default builds, since it would add some bulk and so few people will ever use it. It may therefore be on those of us who are interested in finding efficient ways to deploy IB support. Regards, Jason P.S. Step 1 in my exploration: <<>> /home/bacon 1001 # find /usr/src/ -type f -exec fgrep -l OFED '{}' \; /usr/src/tools/build/options/WITH_OFED /usr/src/etc/rc.d/Makefile /usr/src/share/man/man5/src.conf.5 /usr/src/share/man/man5/pf.conf.5 /usr/src/share/mk/bsd.own.mk /usr/src/contrib/ofed/libmlx4/fixes/lim_qp_resources.patch /usr/src/contrib/ofed/libmlx4/fixes/userspace_dev_lims.patch /usr/src/contrib/ofed/libmlx4/fixes/xrc_consolidated_v2.patch /usr/src/contrib/ofed/libmlx4/fixes/xrc_kernel_user_mismatch.patch /usr/src/contrib/ofed/libsdp/README /usr/src/contrib/ofed/usr.lib/libsdp/Makefile /usr/src/contrib/ofed/libibverbs/fixes/XRC_RCV_QP.patch /usr/src/contrib/ofed/libibverbs/fixes/pthread_cond_t_fields.patch /usr/src/contrib/ofed/libibverbs/fixes/XRC_base_implementation.patch /usr/src/contrib/ofed/management/opensm/doc/opensm_release_notes_openib-1.2.1.txt /usr/src/contrib/ofed/management/opensm/doc/opensm_release_notes_openib-2.0.5.txt /usr/src/contrib/ofed/management/opensm/doc/opensm_release_notes-3.2.txt /usr/src/contrib/ofed/management/opensm/doc/opensm_release_notes_openib-3.0.13.txt /usr/src/contrib/ofed/management/opensm/doc/performance-manager-HOWTO.txt /usr/src/contrib/ofed/management/opensm/doc/opensm_release_notes-3.1.10.txt /usr/src/contrib/ofed/management/opensm/doc/opensm_release_notes-3.1.11.txt /usr/src/contrib/ofed/management/opensm/complib/ChangeLog /usr/src/contrib/ofed/management/infiniband-diags/infiniband-diags.spec.in /usr/src/sys/modules/ibcore/Makefile /usr/src/sys/modules/ipoib/Makefile /usr/src/sys/modules/Makefile /usr/src/sys/modules/mlx4ib/Makefile /usr/src/sys/amd64/conf/NOTES /usr/src/sys/i386/conf/NOTES /usr/src/sys/conf/kern.pre.mk /usr/src/sys/conf/options /usr/src/sys/conf/files /usr/src/sys/sys/socket.h /usr/src/sys/ofed/include/linux/module.h /usr/src/sys/ofed/drivers/infiniband/hw/mlx4/main.c /usr/src/sys/ofed/drivers/net/mlx4/main.c /usr/src/sys/ofed/drivers/net/mlx4/mlx4.h /usr/src/sys/ofed/drivers/net/mlx4/en_main.c /usr/src/usr.bin/netstat/Makefile /usr/src/Makefile.inc1 -- All wars are civil wars, because all men are brothers ... Each one owes infinitely more to the human race than to the particular country in which he was born. -- Francois Fenelon From owner-freebsd-infiniband@freebsd.org Mon Apr 25 22:05:57 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C9CEBB1C9CC for ; Mon, 25 Apr 2016 22:05:57 +0000 (UTC) (envelope-from dpd@dpdtech.com) Received: from mail-pf0-f179.google.com (mail-pf0-f179.google.com [209.85.192.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9D3491F09 for ; Mon, 25 Apr 2016 22:05:57 +0000 (UTC) (envelope-from dpd@dpdtech.com) Received: by mail-pf0-f179.google.com with SMTP id c189so27126960pfb.3 for ; Mon, 25 Apr 2016 15:05:57 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:mime-version:from:in-reply-to:date:cc :message-id:references:to; bh=cqqF6LsfsF3avLCJzvFJCa4aY4YgFfnM/E8QT3lrKa8=; b=dkq3//ybDhCq5Bww1Wwsqkp8L9FMP86jj/QctDk7xvbO14YY7M/hGDH5SgID+EAmiB 1P7BfSm6upMdoPJzAYdvn51THjBoRq9YfIR0V5gZpdQvmi+96eI0qY8Ci/1JPl4FLWKW JPsfp+B7nAQJZr3DjC2C07WEiB/SoxhEVFez2ks0xv5mJuXFHEFOVhHPbQBD34ECShQc on3fXbca7ucDz1uhu72xsCPGTkkPkpDp0q4JgndAQ78gKAQZidNJJS8hbA2kX4COg2O2 R3TLawRdwFVY4Y2lL1qpgb+qiV6bXuwjFFlSemz7dxoTS7JPJG8hCsLSVWIrPWn9lKxJ +Z5w== X-Gm-Message-State: AOPr4FWEPZag9sB3ZD7WXzmnerOPxuaHo7qbnS5pvmArIrvlAZa+HixqN8AhtJ2AC2sMfw== X-Received: by 10.98.29.16 with SMTP id d16mr5443600pfd.142.1461602515909; Mon, 25 Apr 2016 09:41:55 -0700 (PDT) Received: from [192.168.1.248] (173-13-188-45-sfba.hfc.comcastbusiness.net. [173.13.188.45]) by smtp.gmail.com with ESMTPSA id y3sm31377424par.2.2016.04.25.09.41.53 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 25 Apr 2016 09:41:54 -0700 (PDT) Subject: Re: Request updated/unified patch for OFED v3.2 update Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Content-Type: multipart/signed; boundary="Apple-Mail=_D92C6E6B-E216-451C-B815-4693F82F8A63"; protocol="application/pgp-signature"; micalg=pgp-sha512 X-Pgp-Agent: GPGMail 2.6b2 From: David P. Discher In-Reply-To: Date: Mon, 25 Apr 2016 09:41:52 -0700 Cc: Justin Clift , freebsd-infiniband@freebsd.org Message-Id: <3C9A9DDD-C1AE-4B92-A000-137CF17B9F49@dpdtech.com> References: <571B31B6.4070206@selasky.org> <571BC0A6.9070100@selasky.org> <207113AF-4D33-4DD6-8A69-DD9ABD48DB8F@postgresql.org> <571BE5CD.2050401@selasky.org> <4F7D00E4-C784-4CDB-B058-F3F24B1B2F06@dpdtech.com> <571BEEAB.4040602@selasky.org> To: Hans Petter Selasky X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Apr 2016 22:05:57 -0000 --Apple-Mail=_D92C6E6B-E216-451C-B815-4693F82F8A63 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 A couple notes that frustrated the hell out me this weekend: I could not get the modules to compile in-kernel. They have to be = loadable modules. I kept getting linking errors, to what looks like = references in the linux modules. I could not get the patch set to apply cleanly to r298482. It did apply = cleanly to r298518. Patch doesn=92t create directors, so you must create these directories = *before* patch: mkdir -p contrib/ofed/libmlx5 mkdir -p contrib/ofed/libmlx5/config mkdir -p contrib/ofed/libmlx5/debian mkdir -p contrib/ofed/libmlx5/debian/patches mkdir -p contrib/ofed/libmlx5/debian/source mkdir -p contrib/ofed/libmlx5/scripts/expose_libmlx5_headers mkdir -p contrib/ofed/libmlx5/src mkdir -p contrib/ofed/librdmacm/include/infiniband mkdir -p contrib/ofed/usr.lib/libmlx5 mkdir -p sys/modules/mlx5ib If you get some new files in /usr/src, then you missed a directory. The patch command line I used was : patch --remove-empty-files -F 30 -i /z/home/dpd/OFEDv32/patch.diff patch.diff was created with: cat D5790.diff.txt D5791.diff.txt D5792.diff.txt D5793.diff.txt = D5794.diff.txt D5795.diff.txt D5796.diff.txt D5797.diff.txt = D5798.diff.txt D5799.diff.txt > patch.diff Everything compiled, however I messed up my test systems, so I haven=92t = been able to test anything yet. - David P. Discher http://davidpdischer.com/ AIM: DavidDPD | Y!M: daviddpdz Mobile: 408.368.3725 > On Apr 23, 2016, at 2:51 PM, David P. Discher wrote: >=20 > yup. That fixed it =85 compiling now. >=20 > Thanks ! >=20 >=20 > - > David P. Discher > http://davidpdischer.com/ > AIM: DavidDPD | Y!M: daviddpdz > Mobile: 408.368.3725 >=20 >=20 >=20 >> On Apr 23, 2016, at 2:52 PM, Hans Petter Selasky = wrote: >>=20 >> On 04/23/16 23:40, David P. Discher wrote: >>> I had to refresh all the patches, and they finally applied cleanly =85= however make failed: >>>=20 >>> make: "/usr/src/share/mk/src.libnames.mk" line 560: /usr/src: = Missing or incorrect value for LIBMLX5DIR in src.libnames.mk: = contrib/ofed/usr.lib/libmlx5 >>>=20 >>> It looks like "contrib/ofed/usr.lib/libmlx5=94 is completely = missing. >>>=20 >>> Tried to remove libmlx5 (since I don=92t have a mlx5 card) from the = make files, but seems to all over the place. >>>=20 >>=20 >> I think you maybe need to create that directory before applying the = patches. >>=20 >> --HPS >>=20 >=20 --Apple-Mail=_D92C6E6B-E216-451C-B815-4693F82F8A63 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJXHkjQAAoJEEmwU6XuhYWOlrAIAKwb67W3nUr2Tt3AtTK83uVg xyMa0XRZvZjCUi1bKz7v+6vfzdSt2gSE1hfDAmsd9g1wTLK4l4ZlgpZsUrSSlZpe qqhjciLR8axTqTSYhO3bgCPtTSy/uqKbEBuDKGVu2OQh3EmKhyv+sY1zv2b6JIqL JP5kFQAFI4MTYxWfRX682sBagZSBUeLEL2fHpBNXxYdBrgaX1tXQ1GBipjHy+K9B nOT12tPg30YeJKpvMctKZqZMlBOoWs3im9hmiXis/XhFHYfMlDFAF7UDoevNhkDG B1yRi5V9PreNI502P39xqrMt0MgYv2dZELxQ1jFOd2jSkdOFp0w7zkr6HlhNX/I= =4Vbu -----END PGP SIGNATURE----- --Apple-Mail=_D92C6E6B-E216-451C-B815-4693F82F8A63-- From owner-freebsd-infiniband@freebsd.org Tue Apr 26 06:29:36 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E8B1BB1D426 for ; Tue, 26 Apr 2016 06:29:36 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (heidi.turbocat.net [88.198.202.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B03741AEE for ; Tue, 26 Apr 2016 06:29:36 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id B13A31FE023; Tue, 26 Apr 2016 08:29:27 +0200 (CEST) Subject: Re: Request updated/unified patch for OFED v3.2 update To: "David P. Discher" References: <571B31B6.4070206@selasky.org> <571BC0A6.9070100@selasky.org> <207113AF-4D33-4DD6-8A69-DD9ABD48DB8F@postgresql.org> <571BE5CD.2050401@selasky.org> <4F7D00E4-C784-4CDB-B058-F3F24B1B2F06@dpdtech.com> <571BEEAB.4040602@selasky.org> <3C9A9DDD-C1AE-4B92-A000-137CF17B9F49@dpdtech.com> Cc: Justin Clift , freebsd-infiniband@freebsd.org From: Hans Petter Selasky Message-ID: <571F0B8C.3020606@selasky.org> Date: Tue, 26 Apr 2016 08:32:44 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.5.0 MIME-Version: 1.0 In-Reply-To: <3C9A9DDD-C1AE-4B92-A000-137CF17B9F49@dpdtech.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Apr 2016 06:29:37 -0000 On 04/25/16 18:41, David P. Discher wrote: > A couple notes that frustrated the hell out me this weekend: > > I could not get the modules to compile in-kernel. They have to be loadable modules. I kept getting linking errors, to what looks like references in the linux modules. You'll have to use: options LINUXKPI When building all into the kernel. --HPS > > I could not get the patch set to apply cleanly to r298482. It did apply cleanly to r298518. > > Patch doesn’t create directors, so you must create these directories *before* patch: > > mkdir -p contrib/ofed/libmlx5 > mkdir -p contrib/ofed/libmlx5/config > mkdir -p contrib/ofed/libmlx5/debian > mkdir -p contrib/ofed/libmlx5/debian/patches > mkdir -p contrib/ofed/libmlx5/debian/source > mkdir -p contrib/ofed/libmlx5/scripts/expose_libmlx5_headers > mkdir -p contrib/ofed/libmlx5/src > mkdir -p contrib/ofed/librdmacm/include/infiniband > mkdir -p contrib/ofed/usr.lib/libmlx5 > mkdir -p sys/modules/mlx5ib > > > If you get some new files in /usr/src, then you missed a directory. > > The patch command line I used was : > > patch --remove-empty-files -F 30 -i /z/home/dpd/OFEDv32/patch.diff > > patch.diff was created with: > > cat D5790.diff.txt D5791.diff.txt D5792.diff.txt D5793.diff.txt D5794.diff.txt D5795.diff.txt D5796.diff.txt D5797.diff.txt D5798.diff.txt D5799.diff.txt > patch.diff > It might be easier if you have a git checkout of the FreeBSD kernel. git am xxx handles these issues automatically. --HPS From owner-freebsd-infiniband@freebsd.org Tue Apr 26 14:26:31 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 536F0B1DF18 for ; Tue, 26 Apr 2016 14:26:31 +0000 (UTC) (envelope-from Krishna2@chelsio.com) Received: from stargate.chelsio.com (stargate.chelsio.com [12.32.117.8]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 42B6E14DE for ; Tue, 26 Apr 2016 14:26:30 +0000 (UTC) (envelope-from Krishna2@chelsio.com) Received: from nice.asicdesigners.com (nice.asicdesigners.com [10.192.160.7]) by stargate.chelsio.com (8.13.8/8.13.8) with ESMTP id u3QEQT4J028087; Tue, 26 Apr 2016 07:26:30 -0700 Received: from NICE.asicdesigners.com ([fe80::51b2:ba95:9d72:babc]) by nice.asicdesigners.com ([fe80::51b2:ba95:9d72:babc%15]) with mapi id 14.03.0123.003; Tue, 26 Apr 2016 07:26:28 -0700 From: KrishnamRaju ErapaRaju To: Hans Petter Selasky , "freebsd-infiniband@freebsd.org" Subject: RE: OFED v3.2 update Thread-Topic: OFED v3.2 update Thread-Index: AdGfxvy3pMSiVbkRTtKHZB5IpjPLLg== Date: Tue, 26 Apr 2016 14:26:26 +0000 Message-ID: <4EAB2619577FCA4EBCE11F17B7373447C4309255@nice.asicdesigners.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.193.191.70] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Apr 2016 14:26:31 -0000 Hello Hans, I could apply OFED 3.2 patches cleanly to r298617 but failed to build(buil= dworld), below is the compile error:(more error log at the bottom of this m= ail) bmake[5]: bmake[5]: don't know how to make buf.c. Stop I took the latest OFED 3.2 patchsets as of today. And applied the patches w= ith commands: "patch --remove-empty-files -F 30 -i /usr/src/ofed3.2/D5790.d= iff" Here is my config file: diff -r 5615b648049d sys/amd64/conf/GENERIC --- a/sys/amd64/conf/GENERIC Tue Apr 26 05:21:27 2016 +0000 +++ b/sys/amd64/conf/GENERIC Mon Apr 25 15:34:54 2016 +0530 @@ -21,8 +21,9 @@ -makeoptions DEBUG=3D-g # Build kernel with gdb(1) debug = symbols +makeoptions DEBUG=3D"-g -O0" # Build kernel with gdb(1) debug = symbols +options KSTACK_PAGES=3D16 +options KTR +options KTR_COMPILE=3DKTR_SPARE3 +options KTR_MASK=3DKTR_SPARE3 +options KTR_ENTRIES=3D65536 +options BREAK_TO_DEBUGGER +options LINUXKPI -------------------------------COMPILE ERROR LOG-------------------- cc -O2 -pipe -I/usr/src/cddl/lib/libzpool/../../../sys/cddl/compat/openso= laris -I/usr/src/cddl/lib/libzpool/../../compat/opensolaris/include -I/usr/= src/cddl/lib/libzpool/../../compat/opensolaris/lib/libumem -I/usr/src/cddl/= lib/libzpool/../../contrib/opensolaris/lib/libzpool/common -I/usr/src/cddl/= lib/libzpool/../../../sys/cddl/contrib/opensolaris/uts/common/sys -I/usr/sr= c/cddl/lib/libzpool/../../../sys/cddl/contrib/opensolaris/uts/common/fs/zfs= -I/usr/src/cddl/lib/libzpool/../../../sys/cddl/contrib/opensolaris/common/= zfs -I/usr/src/cddl/lib/libzpool/../../../sys/cddl/contrib/opensolaris/uts/= common -I/usr/src/cddl/lib/libzpool/../../contrib/opensolaris/head -I/usr/s= rc/cddl/lib/libzpool/../../lib/libumem -I/usr/src/cddl/lib/libzpool/../../c= ontrib/opensolaris/lib/libnvpair -DWANTS_MUTEX_OWNED -I/usr/src/cddl/lib/li= bzpool/../../../lib/libpthread/thread -I/usr/src/cddl/lib/libzpool/../../..= /lib/libpthread/sys -I/usr/src/cddl/lib/libzpool/../../../lib/libthr/arch/a= md64/include -g -DDEBUG=3D1 -DNEED_SOLARIS_BOOLEAN -MD -MF.depend.dmu_zf= etch.o -MTdmu_zfetch.o -std=3Diso9899:1999 -fstack-protector-strong -Wno-po= inter-sign -Wno-unknown-pragmas -Wno-empty-body -Wno-string-plus-int -Wno-u= nused-const-variable -Wno-tautological-compare -Wno-unused-value -Wno-paren= theses-equality -Wno-unused-function -Wno-enum-conversion -Wno-unused-local= -typedef -Wno-switch -Wno-switch-enum -Wno-knr-promoted-parameter -Wno-pare= ntheses -Qunused-arguments -c /usr/src/cddl/lib/libzpool/../../../sys/cdd= l/contrib/opensolaris/uts/common/fs/zfs/dmu_zfetch.c -o dmu_zfetch.o --- contrib/ofed/usr.lib__L --- bmake[5]: bmake[5]: don't know how to make buf.c. Stop bmake[5]: stopped in /usr/src/contrib/ofed/usr.lib/libmlx5 --- kerberos5/lib__L --- --- asn1_MechTypeList.c --- --- contrib/ofed/usr.lib__L --- Command used for compilation: -> make -j4 buildworld WITH_OFED=3DYES -> make -j4 buildkernel WITH_OFED=3DYES Am I missing something here?, please guide me in successfully building the = kernel/world with OFED 3.2 pathes. Thanks, Krishna. -----Original Message----- From: owner-freebsd-infiniband@freebsd.org [mailto:owner-freebsd-infiniband= @freebsd.org] On Behalf Of Hans Petter Selasky Sent: Thursday, March 31, 2016 3:21 PM To: freebsd-infiniband@freebsd.org Subject: OFED v3.2 update Dear users of infiniband in FreeBSD: Mellanox is working on a big infiniband update towards mellanox OFED v3.2 of the infiniband stack (ibcore), drivers (mlx4/mlx5) and libraries (l= ibrdma, libibverbs ...). Currently there are 10-patches in different areas = of the FreeBSD kernel and userspace which you need to apply to 11-current a= s of today to get a working GENERIC kernel. Compile world and kernel like this after applying the patches below: make buildkernel WITH_OFED=3DYES make buildworld WITH_OFED=3DYES https://reviews.freebsd.org/D5790 https://reviews.freebsd.org/D5791 https://reviews.freebsd.org/D5792 https://reviews.freebsd.org/D5793 https://reviews.freebsd.org/D5794 https://reviews.freebsd.org/D5795 https://reviews.freebsd.org/D5796 https://reviews.freebsd.org/D5797 https://reviews.freebsd.org/D5798 https://reviews.freebsd.org/D5799 The patches are work in progress, though at a level where basic infiniband = is working with MLX5. Known issues: - LINT build is broken (Will be fixed before committing. Waiting for=20 solution from Navdeep) - Need to unload ipoib before mlx5ib. --HPS _______________________________________________ freebsd-infiniband@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-infiniband To unsubscribe, send any mail to "freebsd-infiniband-unsubscribe@freebsd.or= g" From owner-freebsd-infiniband@freebsd.org Tue Apr 26 14:53:28 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 17F05B1C9C6 for ; Tue, 26 Apr 2016 14:53:28 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (heidi.turbocat.net [88.198.202.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D674C101D for ; Tue, 26 Apr 2016 14:53:27 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 075871FE023; Tue, 26 Apr 2016 16:53:24 +0200 (CEST) Subject: Re: OFED v3.2 update To: KrishnamRaju ErapaRaju , "freebsd-infiniband@freebsd.org" References: <4EAB2619577FCA4EBCE11F17B7373447C4309255@nice.asicdesigners.com> From: Hans Petter Selasky Message-ID: <571F81A9.7080006@selasky.org> Date: Tue, 26 Apr 2016 16:56:41 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.5.0 MIME-Version: 1.0 In-Reply-To: <4EAB2619577FCA4EBCE11F17B7373447C4309255@nice.asicdesigners.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Apr 2016 14:53:28 -0000 Hi Krishnam, If you are not using GIT's "am" commend, you'll need to create the following directories beforehand: mkdir -p contrib/ofed/libmlx5 mkdir -p contrib/ofed/libmlx5/config mkdir -p contrib/ofed/libmlx5/debian mkdir -p contrib/ofed/libmlx5/debian/patches mkdir -p contrib/ofed/libmlx5/debian/source mkdir -p contrib/ofed/libmlx5/scripts/expose_libmlx5_headers mkdir -p contrib/ofed/libmlx5/src mkdir -p contrib/ofed/librdmacm/include/infiniband mkdir -p contrib/ofed/usr.lib/libmlx5 mkdir -p sys/modules/mlx5ib --HPS On 04/26/16 16:26, KrishnamRaju ErapaRaju wrote: > > Hello Hans, > > > I could apply OFED 3.2 patches cleanly to r298617 but failed to build(buildworld), below is the compile error:(more error log at the bottom of this mail) > bmake[5]: bmake[5]: don't know how to make buf.c. Stop > > I took the latest OFED 3.2 patchsets as of today. And applied the patches with commands: "patch --remove-empty-files -F 30 -i /usr/src/ofed3.2/D5790.diff" > > Here is my config file: > > diff -r 5615b648049d sys/amd64/conf/GENERIC > --- a/sys/amd64/conf/GENERIC Tue Apr 26 05:21:27 2016 +0000 > +++ b/sys/amd64/conf/GENERIC Mon Apr 25 15:34:54 2016 +0530 > @@ -21,8 +21,9 @@ > -makeoptions DEBUG=-g # Build kernel with gdb(1) debug symbols > +makeoptions DEBUG="-g -O0" # Build kernel with gdb(1) debug symbols > +options KSTACK_PAGES=16 > +options KTR > +options KTR_COMPILE=KTR_SPARE3 > +options KTR_MASK=KTR_SPARE3 > +options KTR_ENTRIES=65536 > +options BREAK_TO_DEBUGGER > +options LINUXKPI > From owner-freebsd-infiniband@freebsd.org Wed Apr 27 17:14:11 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 06543B1E204 for ; Wed, 27 Apr 2016 17:14:11 +0000 (UTC) (envelope-from bacon@uwm.edu) Received: from mail.tds.net (smtp-fo.tds.net [64.8.70.105]) by mx1.freebsd.org (Postfix) with ESMTP id CC3CE198E for ; Wed, 27 Apr 2016 17:14:10 +0000 (UTC) (envelope-from bacon@uwm.edu) X_CMAE_Category: , , X-CNFS-Analysis: v=2.2 cv=MZAugN3f c=1 sm=1 tr=0 a=oQHqK7Vjp8f9DEA8UMKPjA==:117 a=oQHqK7Vjp8f9DEA8UMKPjA==:17 a=N659UExz7-8A:10 a=6I5d2MoRAAAA:8 a=ndaoGXS1AAAA:8 a=Rj8nKAVhZKzZItEUdpkA:9 a=KnYvbJ33n62V4Fv3:21 a=PkixxCj24RNiXX4w:21 X-CM-Score: 0 X-Scanned-by: Cloudmark Authority Engine X-Authed-Username: andiYWNvbkB0ZHMubmV0 Authentication-Results: smtp02.tds.cmh.synacor.com smtp.user=jwbacon@tds.net; auth=pass (LOGIN) Received: from [64.83.177.9] ([64.83.177.9:17597] helo=imacbsd.acadix.biz) by mail.tds.net (envelope-from ) (ecelerity 3.6.5.45644 r(Core:3.6.5.0)) with ESMTPSA (cipher=AES128-SHA) id 0B/1D-09870-AAEE0275; Wed, 27 Apr 2016 12:54:03 -0400 Subject: Re: OFED v3.2 update To: Chris Reeves , "freebsd-infiniband@freebsd.org" References: <56FCF309.1070406@selasky.org> <1459534095550.93717@spectralogic.com> From: Jason Bacon Organization: UW - Milwaukee Message-ID: <5720EEA9.6030203@uwm.edu> Date: Wed, 27 Apr 2016 11:54:01 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.7.1 MIME-Version: 1.0 In-Reply-To: <1459534095550.93717@spectralogic.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Apr 2016 17:14:11 -0000 This is encouraging... On our CentOS 6.7 cluster, what we're seeing is comparable to the Ubuntu performance you reported earlier. ( Best of 3 runs between two idle compute nodes on the same leaf switch shown below. ) It appears that we may be able to improve NFS performance using a FreeBSD/FreeNAS file server in the near future. Regards, Jason Linux compute-083. bacon ~ 335: iperf -c compute-082-ib0 ------------------------------------------------------------ Client connecting to compute-082-ib0, TCP port 5001 TCP window size: 645 KByte (default) ------------------------------------------------------------ [ 3] local 10.2.11.109 port 47470 connected with 10.2.11.108 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.0 sec 22.3 GBytes 19.2 Gbits/sec On 04/01/16 13:08, Chris Reeves wrote: > Thank you Hans, > > Using these patches, I was able to get my IPOIB performance up to 25 Gb/s (vs 10 Gb/s on 11-current without the patches). > > Out of curiosity, do you have any plans to MFC these to stable-10? > > Thanks, > Chris > ________________________________________ > From: owner-freebsd-infiniband@freebsd.org on behalf of Hans Petter Selasky > Sent: Thursday, March 31, 2016 3:51 AM > To: freebsd-infiniband@freebsd.org > Subject: OFED v3.2 update > > Dear users of infiniband in FreeBSD: > > > Mellanox is working on a big infiniband update towards mellanox OFED > v3.2 of the infiniband stack (ibcore), drivers (mlx4/mlx5) and libraries > (librdma, libibverbs ...). Currently there are 10-patches in different > areas of the FreeBSD kernel and userspace which you need to apply to > 11-current as of today to get a working GENERIC kernel. > > Compile world and kernel like this after applying the patches below: > > make buildkernel WITH_OFED=YES > make buildworld WITH_OFED=YES > > > https://reviews.freebsd.org/D5790 > https://reviews.freebsd.org/D5791 > https://reviews.freebsd.org/D5792 > https://reviews.freebsd.org/D5793 > https://reviews.freebsd.org/D5794 > https://reviews.freebsd.org/D5795 > https://reviews.freebsd.org/D5796 > https://reviews.freebsd.org/D5797 > https://reviews.freebsd.org/D5798 > https://reviews.freebsd.org/D5799 > > > The patches are work in progress, though at a level where basic > infiniband is working with MLX5. > > > Known issues: > - LINT build is broken (Will be fixed before committing. Waiting for > solution from Navdeep) > - Need to unload ipoib before mlx5ib. > > --HPS > _______________________________________________ > freebsd-infiniband@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-infiniband > To unsubscribe, send any mail to "freebsd-infiniband-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-infiniband@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-infiniband > To unsubscribe, send any mail to "freebsd-infiniband-unsubscribe@freebsd.org" -- ------------------------------------- Jason W. Bacon Systems Manager Research Computing Support University of Wisconsin Milwaukee bacon@uwm.edu ------------------------------------- From owner-freebsd-infiniband@freebsd.org Thu Apr 28 13:09:03 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6780DB1DF9B for ; Thu, 28 Apr 2016 13:09:03 +0000 (UTC) (envelope-from Krishna2@chelsio.com) Received: from stargate.chelsio.com (stargate.chelsio.com [12.32.117.8]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4ACAC1F32 for ; Thu, 28 Apr 2016 13:09:03 +0000 (UTC) (envelope-from Krishna2@chelsio.com) Received: from nice.asicdesigners.com (nice.asicdesigners.com [10.192.160.7]) by stargate.chelsio.com (8.13.8/8.13.8) with ESMTP id u3SD8u89030144; Thu, 28 Apr 2016 06:09:02 -0700 Received: from NICE.asicdesigners.com ([fe80::51b2:ba95:9d72:babc]) by nice.asicdesigners.com ([fe80::51b2:ba95:9d72:babc%15]) with mapi id 14.03.0123.003; Thu, 28 Apr 2016 06:08:56 -0700 From: KrishnamRaju ErapaRaju To: Hans Petter Selasky , "freebsd-infiniband@freebsd.org" Subject: RE: OFED v3.2 update Thread-Topic: OFED v3.2 update Thread-Index: AdGhTtXn1byEZBF6QiiKvqfL/lDwRw== Date: Thu, 28 Apr 2016 13:08:53 +0000 Message-ID: <4EAB2619577FCA4EBCE11F17B7373447C4309697@nice.asicdesigners.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-originating-ip: [10.193.191.70] Content-Type: multipart/mixed; boundary="_002_4EAB2619577FCA4EBCE11F17B7373447C4309697niceasicdesigne_" MIME-Version: 1.0 X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Apr 2016 13:09:03 -0000 --_002_4EAB2619577FCA4EBCE11F17B7373447C4309697niceasicdesigne_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hello Hans,=20 Thanks, now I am able to build successfully. ("sys/dev/mlx5/mlx5_ib" also need to be created along with your given list = of directories) Are these latest patches in mailing list are final now? It looks like TCP socket logic(needed for IWARP driver) is missing in the = latest patchsets, for example: In D5791.diff Index: sys/ofed/drivers/infiniband/core/cma.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- sys/ofed/drivers/infiniband/core/cma.c +++ sys/ofed/drivers/infiniband/core/cma.c @@ -2571,79 +2916,74 @@ return ret; } -static int cma_get_tcp_port(struct rdma_id_private *id_priv) +static struct idr *cma_select_inet_ps(struct rdma_id_private *id_priv) { - int ret; - int size; - struct socket *sock; - - ret =3D sock_create_kern(AF_INET, SOCK_STREAM, IPPROTO_TCP, &sock); - if (ret) - return ret; ---------------------------------------------------------------------------= -------------------------------------------- Also, when I try to run rping(after adding the missing TCP socket logic bac= k,attached) I am facing the "poll error -78" error.=20 Were you able to run rping with IBOE successfully? Looks like the legacy behavior of ibv_exp_poll_cq() got changed. Will try t= o root cause this issue. - - - Here are the steps I followed(for those who use Mercurial Version control S= W): -> Create below directories: mkdir -p contrib/ofed/libmlx5 mkdir -p contrib/ofed/libmlx5/config mkdir -p contrib/ofed/libmlx5/debian mkdir -p contrib/ofed/libmlx5/debian/patches mkdir -p contrib/ofed/libmlx5/debian/source mkdir -p contrib/ofed/libmlx5/scripts/expose_libmlx5_headers mkdir -p contrib/ofed/libmlx5/src mkdir -p contrib/ofed/librdmacm/include/infiniband mkdir -p contrib/ofed/usr.lib/libmlx5 mkdir -p sys/modules/mlx5ib mkdir -p sys/dev/mlx5/mlx5_ib -> Get the latest patches from "https://lists.freebsd.org/pipermail/freebsd= -infiniband/2016-March/000190.html" -> Apply patches: patch --remove-empty-files -F 30 -i /bld_kit/ofed3.2_v3/D5790.diff patch --remove-empty-files -F 30 -i /bld_kit/ofed3.2_v3/D5791.diff patch --remove-empty-files -F 30 -i /bld_kit/ofed3.2_v3/D5792.diff patch --remove-empty-files -F 30 -i /bld_kit/ofed3.2_v3/D5793.diff patch --remove-empty-files -F 30 -i /bld_kit/ofed3.2_v3/D5794.diff patch --remove-empty-files -F 30 -i /bld_kit/ofed3.2_v3/D5795.diff patch --remove-empty-files -F 30 -i /bld_kit/ofed3.2_v3/D5796.diff patch --remove-empty-files -F 30 -i /bld_kit/ofed3.2_v3/D5797.diff patch --remove-empty-files -F 30 -i /bld_kit/ofed3.2_v3/D5798.diff patch --remove-empty-files -F 30 -i /bld_kit/ofed3.2_v3/D5799.diff Build process: make -j8 buildworld WITH_OFED=3DYES make -j8 buildkernel WITH_OFED=3DYES make installkernel make installworld Thanks, Krishna. -----Original Message----- From: Hans Petter Selasky [mailto:hps@selasky.org]=20 Sent: Tuesday, April 26, 2016 8:27 PM To: KrishnamRaju ErapaRaju ; freebsd-infiniband@freeb= sd.org Subject: Re: OFED v3.2 update Hi Krishnam, If you are not using GIT's "am" commend, you'll need to create the followin= g directories beforehand: mkdir -p contrib/ofed/libmlx5 mkdir -p contrib/ofed/libmlx5/config mkdir -p contrib/ofed/libmlx5/debian mkdir -p contrib/ofed/libmlx5/debian/patches mkdir -p contrib/ofed/libmlx5/debian/source mkdir -p contrib/ofed/libmlx5/scripts/expose_libmlx5_headers mkdir -p contrib/ofed/libmlx5/src mkdir -p contrib/ofed/librdmacm/include/infiniband mkdir -p contrib/ofed/usr.lib/libmlx5 mkdir -p sys/modules/mlx5ib --HPS On 04/26/16 16:26, KrishnamRaju ErapaRaju wrote: > > Hello Hans, > > > I could apply OFED 3.2 patches cleanly to r298617 but failed to build(bu= ildworld), below is the compile error:(more error log at the bottom of this= mail) > bmake[5]: bmake[5]: don't know how to make buf.c. Stop > > I took the latest OFED 3.2 patchsets as of today. And applied the patches= with commands: "patch --remove-empty-files -F 30 -i /usr/src/ofed3.2/D5790= .diff" > > Here is my config file: > > diff -r 5615b648049d sys/amd64/conf/GENERIC > --- a/sys/amd64/conf/GENERIC Tue Apr 26 05:21:27 2016 +0000 > +++ b/sys/amd64/conf/GENERIC Mon Apr 25 15:34:54 2016 +0530 > @@ -21,8 +21,9 @@ > -makeoptions DEBUG=3D-g # Build kernel with gdb(1) debu= g symbols > +makeoptions DEBUG=3D"-g -O0" # Build kernel with gdb(1) debu= g symbols > +options KSTACK_PAGES=3D16 > +options KTR > +options KTR_COMPILE=3DKTR_SPARE3 > +options KTR_MASK=3DKTR_SPARE3 > +options KTR_ENTRIES=3D65536 > +options BREAK_TO_DEBUGGER > +options LINUXKPI > --_002_4EAB2619577FCA4EBCE11F17B7373447C4309697niceasicdesigne_ Content-Type: application/octet-stream; name="tcpsocket.patch" Content-Description: tcpsocket.patch Content-Disposition: attachment; filename="tcpsocket.patch"; size=2826; creation-date="Thu, 28 Apr 2016 13:06:33 GMT"; modification-date="Wed, 27 Apr 2016 04:54:43 GMT" Content-Transfer-Encoding: base64 ZGlmZiAtciBiY2Q5M2NiNjczYjggc3lzL29mZWQvZHJpdmVycy9pbmZpbmliYW5kL2NvcmUvY21h LmMKLS0tIGEvc3lzL29mZWQvZHJpdmVycy9pbmZpbmliYW5kL2NvcmUvY21hLmMJV2VkIEFwciAy NyAwNTowODozOSAyMDE2ICswNTMwCisrKyBiL3N5cy9vZmVkL2RyaXZlcnMvaW5maW5pYmFuZC9j b3JlL2NtYS5jCVdlZCBBcHIgMjcgMTA6MjM6MDIgMjAxNiArMDUzMApAQCAtNzIsNiArNzIsMTEg QEAKIG1vZHVsZV9wYXJhbV9uYW1lZChkZWZfcHJlYzJzbCwgZGVmX3ByZWMyc2wsIGludCwgMDY0 NCk7CiBNT0RVTEVfUEFSTV9ERVNDKGRlZl9wcmVjMnNsLCAiRGVmYXVsdCB2YWx1ZSBmb3IgU0wg cHJpb3JpdHkgd2l0aCBSb0NFLiBWYWxpZCB2YWx1ZXMgMCAtIDciKTsKIAorc3RhdGljIGludCB1 bmlmeV90Y3BfcG9ydF9zcGFjZSA9IDE7Cittb2R1bGVfcGFyYW0odW5pZnlfdGNwX3BvcnRfc3Bh Y2UsIGludCwgMDY0NCk7CitNT0RVTEVfUEFSTV9ERVNDKHVuaWZ5X3RjcF9wb3J0X3NwYWNlLCAi VW5pZnkgdGhlIGhvc3QgVENQIGFuZCBSRE1BIHBvcnQgIgorCQkic3BhY2UgYWxsb2NhdGlvbiAo ZGVmYXVsdD0xKSIpOworCiBzdGF0aWMgaW50IGRlYnVnX2xldmVsID0gMDsKICNkZWZpbmUgY21h X3ByKGxldmVsLCBwcml2LCBmb3JtYXQsIGFyZy4uLikJCVwKIAlwcmludGsobGV2ZWwgIkNNQTog JXA6ICVzOiAiIGZvcm1hdCwgKChzdHJ1Y3QgcmRtYV9pZF9wcml2ICopIHByaXYpICwgX19mdW5j X18sICMjIGFyZykKQEAgLTE3MCw2ICsxNzUsMTAgQEAKIHsKIAlyZXR1cm4gY21hX2Rldi0+ZGVm YXVsdF9naWRfdHlwZTsKIH0KK3N0YXRpYyBpbmxpbmUgaW50IGlwX2FkZHJfc2l6ZShzdHJ1Y3Qg c29ja2FkZHIgKmFkZHIpCit7CisJcmV0dXJuIGFkZHItPnNhX2ZhbWlseSA9PSBBRl9JTkVUNiA/ c2l6ZW9mKHN0cnVjdCBzb2NrYWRkcl9pbjYpIDogc2l6ZW9mKHN0cnVjdCBzb2NrYWRkcl9pbik7 Cit9CiAKIHZvaWQgY21hX3NldF9kZWZhdWx0X2dpZF90eXBlKHN0cnVjdCBjbWFfZGV2aWNlICpj bWFfZGV2LAogCQkJICAgICAgZW51bSBpYl9naWRfdHlwZSBkZWZhdWx0X2dpZF90eXBlKQpAQCAt MTI5OSw2ICsxMzA4LDkgQEAKIAkJa2ZyZWUoYmluZF9saXN0KTsKIAl9CiAJbXV0ZXhfdW5sb2Nr KCZsb2NrKTsKKworCWlmICgoaWRfcHJpdi0+c29jayE9IE5VTEwpICYmIChpZF9wcml2LT5zb2Nr LT5zb19jb3VudCA+IDApKQorCQlzb2NrX3JlbGVhc2UoaWRfcHJpdi0+c29jayk7CiB9CiAKIHN0 YXRpYyB2b2lkIGNtYV9sZWF2ZV9tY19ncm91cHMoc3RydWN0IHJkbWFfaWRfcHJpdmF0ZSAqaWRf cHJpdikKQEAgLTIwMTAsNiArMjAyMiw3IEBACiAJZGV2X2lkX3ByaXYgPSBjb250YWluZXJfb2Yo aWQsIHN0cnVjdCByZG1hX2lkX3ByaXZhdGUsIGlkKTsKIAogCWRldl9pZF9wcml2LT5zdGF0ZSA9 IFJETUFfQ01fQUREUl9CT1VORDsKKwlkZXZfaWRfcHJpdi0+c29jayA9IGlkX3ByaXYtPnNvY2s7 CiAJbWVtY3B5KGNtYV9zcmNfYWRkcihkZXZfaWRfcHJpdiksIGNtYV9zcmNfYWRkcihpZF9wcml2 KSwKIAkgICAgICAgcmRtYV9hZGRyX3NpemUoY21hX3NyY19hZGRyKGlkX3ByaXYpKSk7CiAKQEAg LTI5MTYsMTMgKzI5MjksNTcgQEAKIAlyZXR1cm4gcmV0OwogfQogCitzdGF0aWMgaW50IGNtYV9n ZXRfdGNwX3BvcnQoc3RydWN0IHJkbWFfaWRfcHJpdmF0ZSAqaWRfcHJpdikKK3sKKwlpbnQgcmV0 OworCWludCBzaXplOworCXN0cnVjdCBzb2NrZXQgKnNvY2s7CisKKwlyZXQgPSBzb2NrX2NyZWF0 ZV9rZXJuKEFGX0lORVQsIFNPQ0tfU1RSRUFNLCBJUFBST1RPX1RDUCwgJnNvY2spOworCWlmIChy ZXQpCisJCXJldHVybiByZXQ7CisjaWZkZWYgX19saW51eF9fCisJcmV0ID0gc29jay0+b3BzLT5i aW5kKHNvY2ssCisJCQkoc3RydWN0IHNvY2thZGRyICopICZpZF9wcml2LT5pZC5yb3V0ZS5hZGRy LnNyY19hZGRyLAorCQkJaXBfYWRkcl9zaXplKChzdHJ1Y3Qgc29ja2FkZHIgKikgJmlkX3ByaXYt PmlkLnJvdXRlLmFkZHIuc3JjX2FkZHIpKTsKKyNlbHNlCisJcmV0ID0gLXNvYmluZChzb2NrLAor CQkJKHN0cnVjdCBzb2NrYWRkciAqKSZpZF9wcml2LT5pZC5yb3V0ZS5hZGRyLnNyY19hZGRyLAor CQkJY3VydGhyZWFkKTsKKyNlbmRpZgorCWlmIChyZXQpIHsKKwkJc29ja19yZWxlYXNlKHNvY2sp OworCQlyZXR1cm4gcmV0OworCX0KKworCXNpemUgPSBpcF9hZGRyX3NpemUoKHN0cnVjdCBzb2Nr YWRkciAqKSAmaWRfcHJpdi0+aWQucm91dGUuYWRkci5zcmNfYWRkcik7CisJcmV0ID0gc29ja19n ZXRuYW1lKHNvY2ssCisJCQkoc3RydWN0IHNvY2thZGRyICopICZpZF9wcml2LT5pZC5yb3V0ZS5h ZGRyLnNyY19hZGRyLAorCQkJJnNpemUsIDApOworCWlmIChyZXQpIHsKKwkJc29ja19yZWxlYXNl KHNvY2spOworCQlyZXR1cm4gcmV0OworCX0KKworCWlkX3ByaXYtPnNvY2sgPSBzb2NrOworCXJl dHVybiAwOworfQorCiBzdGF0aWMgc3RydWN0IGlkciAqY21hX3NlbGVjdF9pbmV0X3BzKHN0cnVj dCByZG1hX2lkX3ByaXZhdGUgKmlkX3ByaXYpCiB7CisJaW50IHJldDsKKwogCXN3aXRjaCAoaWRf cHJpdi0+aWQucHMpIHsKIAljYXNlIFJETUFfUFNfU0RQOgogCQlyZXR1cm4gJnNkcF9wczsKIAlj YXNlIFJETUFfUFNfVENQOgorCQlpZiAodW5pZnlfdGNwX3BvcnRfc3BhY2UpIHsKKwkJCXJldCA9 IGNtYV9nZXRfdGNwX3BvcnQoaWRfcHJpdik7CisJCQlpZiAocmV0KQorCQkJCXJldHVybiBOVUxM OworCQl9CiAJCXJldHVybiAmdGNwX3BzOworCQogCWNhc2UgUkRNQV9QU19VRFA6CiAJCXJldHVy biAmdWRwX3BzOwogCWNhc2UgUkRNQV9QU19JUE9JQjoK --_002_4EAB2619577FCA4EBCE11F17B7373447C4309697niceasicdesigne_-- From owner-freebsd-infiniband@freebsd.org Thu Apr 28 13:19:15 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 48A53B1F13C for ; Thu, 28 Apr 2016 13:19:15 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (mail.turbocat.net [IPv6:2a01:4f8:d16:4514::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 140961254 for ; Thu, 28 Apr 2016 13:19:15 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 9FABA1FE023; Thu, 28 Apr 2016 15:19:11 +0200 (CEST) Subject: Re: OFED v3.2 update To: KrishnamRaju ErapaRaju , "freebsd-infiniband@freebsd.org" References: <4EAB2619577FCA4EBCE11F17B7373447C4309697@nice.asicdesigners.com> From: Hans Petter Selasky Message-ID: <57220E94.6050805@selasky.org> Date: Thu, 28 Apr 2016 15:22:28 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.5.0 MIME-Version: 1.0 In-Reply-To: <4EAB2619577FCA4EBCE11F17B7373447C4309697@nice.asicdesigners.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Apr 2016 13:19:15 -0000 Hi KrishnamRaju, On 04/28/16 15:08, KrishnamRaju ErapaRaju wrote: > Hello Hans, > > Thanks, now I am able to build successfully. > ("sys/dev/mlx5/mlx5_ib" also need to be created along with your given list of directories) > > Are these latest patches in mailing list are final now? No. > > It looks like TCP socket logic(needed for IWARP driver) is missing in the latest patchsets, for example: I'll have a look at it. > In D5791.diff > > Index: sys/ofed/drivers/infiniband/core/cma.c > =================================================================== > --- sys/ofed/drivers/infiniband/core/cma.c > +++ sys/ofed/drivers/infiniband/core/cma.c > @@ -2571,79 +2916,74 @@ > return ret; > } > > -static int cma_get_tcp_port(struct rdma_id_private *id_priv) > +static struct idr *cma_select_inet_ps(struct rdma_id_private *id_priv) > { > - int ret; > - int size; > - struct socket *sock; > - > - ret = sock_create_kern(AF_INET, SOCK_STREAM, IPPROTO_TCP, &sock); > - if (ret) > - return ret; > ----------------------------------------------------------------------------------------------------------------------- > > Also, when I try to run rping(after adding the missing TCP socket logic back,attached) I am facing the "poll error -78" error. > Were you able to run rping with IBOE successfully? With Mellanox cards rping works. It might be a verbs issue. Error code 76 is "ENOSYS Function not implemented". Just grep for that in ibcore, and I guess you'll find it. > Looks like the legacy behavior of ibv_exp_poll_cq() got changed. Will try to root cause this issue. > Thank you. I'll integrate your attached patch. --HPS From owner-freebsd-infiniband@freebsd.org Thu Apr 28 14:58:30 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B35DAB1F889 for ; Thu, 28 Apr 2016 14:58:30 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (mail.turbocat.net [IPv6:2a01:4f8:d16:4514::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7F66D199E for ; Thu, 28 Apr 2016 14:58:30 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 93D551FE023; Thu, 28 Apr 2016 16:58:27 +0200 (CEST) Subject: Re: OFED v3.2 update To: KrishnamRaju ErapaRaju , "freebsd-infiniband@freebsd.org" References: <4EAB2619577FCA4EBCE11F17B7373447C4309697@nice.asicdesigners.com> From: Hans Petter Selasky Message-ID: <572225D8.8050501@selasky.org> Date: Thu, 28 Apr 2016 17:01:44 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.5.0 MIME-Version: 1.0 In-Reply-To: <4EAB2619577FCA4EBCE11F17B7373447C4309697@nice.asicdesigners.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Apr 2016 14:58:30 -0000 On 04/28/16 15:08, KrishnamRaju ErapaRaju wrote: > Hello Hans, > > Thanks, now I am able to build successfully. > ("sys/dev/mlx5/mlx5_ib" also need to be created along with your given list of directories) > > Are these latest patches in mailing list are final now? > > It looks like TCP socket logic(needed for IWARP driver) is missing in the latest patchsets, for example: > > In D5791.diff FYI: I've now updated D5791 and D5792. Thank you! --HPS From owner-freebsd-infiniband@freebsd.org Fri Apr 29 13:31:23 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0E3A8B21931 for ; Fri, 29 Apr 2016 13:31:23 +0000 (UTC) (envelope-from Krishna2@chelsio.com) Received: from stargate.chelsio.com (stargate.chelsio.com [12.32.117.8]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D017B1735 for ; Fri, 29 Apr 2016 13:31:22 +0000 (UTC) (envelope-from Krishna2@chelsio.com) Received: from nice.asicdesigners.com (nice.asicdesigners.com [10.192.160.7]) by stargate.chelsio.com (8.13.8/8.13.8) with ESMTP id u3TDVKXn015223; Fri, 29 Apr 2016 06:31:21 -0700 Received: from NICE.asicdesigners.com ([fe80::51b2:ba95:9d72:babc]) by nice.asicdesigners.com ([fe80::51b2:ba95:9d72:babc%15]) with mapi id 14.03.0123.003; Fri, 29 Apr 2016 06:31:20 -0700 From: KrishnamRaju ErapaRaju To: Hans Petter Selasky , "freebsd-infiniband@freebsd.org" Subject: RE: OFED v3.2 update Thread-Topic: OFED v3.2 update Thread-Index: AQHRoV5yurlUkdpCvka2WceMvz4jq5+g7Uig Date: Fri, 29 Apr 2016 13:31:19 +0000 Message-ID: <4EAB2619577FCA4EBCE11F17B7373447C43098E9@nice.asicdesigners.com> References: <4EAB2619577FCA4EBCE11F17B7373447C4309697@nice.asicdesigners.com> <572225D8.8050501@selasky.org> In-Reply-To: <572225D8.8050501@selasky.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.193.191.70] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Apr 2016 13:31:23 -0000 Thank you very much Hans. I am yet to root cause " ibv_exp_poll_cq()" error(ENOSYS). I will get back = to you soon with the update on this. May I know( in gist) about adding experimental verbs(like ibv_exp_poll_cq)= in recent patches. Like rping, do the RDMA applications(already written with old ib verbs, not= experimental) also needs changes? Thanks, Krishna. -----Original Message----- From: Hans Petter Selasky [mailto:hps@selasky.org]=20 Sent: Thursday, April 28, 2016 8:32 PM To: KrishnamRaju ErapaRaju ; freebsd-infiniband@freeb= sd.org Subject: Re: OFED v3.2 update On 04/28/16 15:08, KrishnamRaju ErapaRaju wrote: > Hello Hans, > > Thanks, now I am able to build successfully. > ("sys/dev/mlx5/mlx5_ib" also need to be created along with your given lis= t of directories) > > Are these latest patches in mailing list are final now? > > It looks like TCP socket logic(needed for IWARP driver) is missing in th= e latest patchsets, for example: > > In D5791.diff FYI: I've now updated D5791 and D5792. Thank you! --HPS From owner-freebsd-infiniband@freebsd.org Fri Apr 29 13:36:10 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 826FEB21B03 for ; Fri, 29 Apr 2016 13:36:10 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (heidi.turbocat.net [88.198.202.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BC4A1B15 for ; Fri, 29 Apr 2016 13:36:09 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id E4DA71FE022; Fri, 29 Apr 2016 15:36:07 +0200 (CEST) Subject: Re: OFED v3.2 update To: KrishnamRaju ErapaRaju , "freebsd-infiniband@freebsd.org" References: <4EAB2619577FCA4EBCE11F17B7373447C4309697@nice.asicdesigners.com> <572225D8.8050501@selasky.org> <4EAB2619577FCA4EBCE11F17B7373447C43098E9@nice.asicdesigners.com> From: Hans Petter Selasky Message-ID: <5723640D.5020201@selasky.org> Date: Fri, 29 Apr 2016 15:39:25 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.5.0 MIME-Version: 1.0 In-Reply-To: <4EAB2619577FCA4EBCE11F17B7373447C43098E9@nice.asicdesigners.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Apr 2016 13:36:10 -0000 On 04/29/16 15:31, KrishnamRaju ErapaRaju wrote: > Thank you very much Hans. > I am yet to root cause " ibv_exp_poll_cq()" error(ENOSYS). I will get back to you soon with the update on this. > > May I know( in gist) about adding experimental verbs(like ibv_exp_poll_cq) in recent patches. > > Like rping, do the RDMA applications(already written with old ib verbs, not experimental) also needs changes? > Hi, libibvers now has a symbol map so that you can choose the version you want to interface. The library should be binary compatible to the old apps. About the experimental verbs, they come from Mellanox and are not upstream Linux yet. This might be a showstopper :-) --HPS From owner-freebsd-infiniband@freebsd.org Fri Apr 29 21:09:25 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 89EAAB21005 for ; Fri, 29 Apr 2016 21:09:25 +0000 (UTC) (envelope-from bacon4000@gmail.com) Received: from mail-ig0-x22d.google.com (mail-ig0-x22d.google.com [IPv6:2607:f8b0:4001:c05::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4484A1230 for ; Fri, 29 Apr 2016 21:09:25 +0000 (UTC) (envelope-from bacon4000@gmail.com) Received: by mail-ig0-x22d.google.com with SMTP id bi2so34928405igb.0 for ; Fri, 29 Apr 2016 14:09:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=xi1rnlqL41J0qTNXRd7sUW0HpsnCqueKqcaj2Kg3WH8=; b=nRH261wUIqJyGtN2QqWBvC7KWy8AU9dB1LtuskENLW22DuyB/zzWlfl+iL7z/L+m/j jwgdq9wwK/7Z/RIqHG5V9DZSfSvC4S3zryiKMhIXXLv664eSVSC7TtE69e4VgQi8qq/R vQWXZz8fiXwhoXlYD41wQUU7PE8jHzHMaNO78PPySMqWvTmQB7mjXUKvtge/jVTrl89b jQ/hryP/U2c44WLbn6PmsksCn7J7TU9rgcczdGBF7i6zolYpzY/IJ++98kO80KuFRs6r DUI7pmnsdTgPsfd+e4EsW/09ZA6zwuLHw+ZniuEnTe7n2+S6jU756DgflmtnSJRukmDh kfqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=xi1rnlqL41J0qTNXRd7sUW0HpsnCqueKqcaj2Kg3WH8=; b=i/fVy+Q/mQiaGDYAjzIlvZaRU22Ajwjf7lIW72C1ZWYGEHdcWLE1PD0kmxqHgsCDKT JBaD8S0e3gpMGU2GUsa0I6Tec5/PNGQzxSE1V5dgaR/F5CoKNDi7Dm2zU0mDhydLDroL lWN7OHG2JQVA/vxuu70gnHx2nJxBacsuCmXGLFHmpNLq/nquUaiozQGlW3IM7XUAh+1w UFE+Tmi1/pxwH/i9C/q19sn8VAuePjlcx0Zq4kf5Ffer62Ju3uMpIm1O3eYd410Dg55Z 81dzczGQxH3ILbxcnKv/0S8ucpxAQr6VqEtOpgXa9NXg5UXGSUdzTBVPtVVZK4jTD2Cs QyCA== X-Gm-Message-State: AOPr4FW5LEq/sn7W8azxoBt2Ku0grx5rMlRgrwNeccDQeW3J/1Dp8gua2nUcXCzqHJ914g== X-Received: by 10.50.10.165 with SMTP id j5mr7017173igb.29.1461964164349; Fri, 29 Apr 2016 14:09:24 -0700 (PDT) Received: from oyster.jbacon.dyndns.org ([64.83.177.9]) by smtp.gmail.com with ESMTPSA id e130sm11256626iof.17.2016.04.29.14.09.23 (version=TLSv1/SSLv3 cipher=OTHER); Fri, 29 Apr 2016 14:09:23 -0700 (PDT) Subject: Re: Initial ISO's ready for testing FreeNAS 9.10 with Mellanox cards To: Justin Clift References: <65B3A04E-53D0-49CB-8DE0-FDE0D8F66B26@postgresql.org> <571E2BA5.5080500@gmail.com> <157d89e5f246d88cb08fa43f2fbf690e@postgresql.org> Cc: freebsd-infiniband@freebsd.org From: Jason Bacon Message-ID: <0567682d-eaa4-332c-d6e6-f267f113f089@gmail.com> Date: Fri, 29 Apr 2016 16:09:21 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.0 MIME-Version: 1.0 In-Reply-To: <157d89e5f246d88cb08fa43f2fbf690e@postgresql.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Apr 2016 21:09:25 -0000 Unfortunately, the installer (ipoib image) is failing on our PowerEdge R730XD. Abbreviated output below: active set on mfid0 No handlers could be found for logger "freenaOS.Configuration" Traceback freenas-install, line 56 if installer.GetPackages() != True Instaler.py, line 857 pkgFile = self._conf.FindPackageFile(pkg, handler=get_file_handler) Configuration.py, line 1109 intr_ok = True Configuration.py, line 638 raise url_exc urllib2.URLError: urlopen error errno 8 hostname nor servname provided, or not known installation on mfid0 has failed This happens about 10 seconds after selecting a boot disk and hitting OK. It didn't get to network configuration or anything else, so the urlopen error seems odd. Note, however, that I was also unable to install FreeBSD with UFS on this system due to the use of the MFI RAID driver. FreeBSD is on the cusp of moving from MFI to MRSAS, so very new LSI controllers aren't supported by MFI, which is still the default. If I install with ZFS, it manages to limp through the installation, but RAID performance is terrible with the MFI driver. After switching to MRSAS, performance is very good. Cheers, Jason On 04/25/16 09:46, Justin Clift wrote: > Awesome, please do. :) > > Noticed your post on the FreeNAS forum about this as well, so I've > posted info there too. The post is in moderation though, so hopefully > someone will approve it and others will test it too. :) > > As a side note, I've been able to do basic interoperability testing of > the Ethernet mode ISO FreeNAS build with a Windows client (also > running ConnectX card). Trying the same thing in IPoIB mode isn't > working though. Seems like driver problems on the Windows side. :( > > Currently seeking assistance with that through the Mellanox Community > forum (don't laugh... I'm not a Windows guy! :>). It's probably just > something simple. Hopefully. ;) > > + Justin > > > On 2016-04-25 15:37, Jason Bacon wrote: >> Justin, >> >> This is *exactly* what I'm interested in doing right now. In fact, I >> had been exploring how to do a FreeNAS build with IB support. Thanks >> a million for providing the ISOs. >> >> I can test with Infinihost and ConnectX-3. >> >> Jason >> >> On 04/25/16 09:03, Justin Clift wrote: >>> This is semi-off-topic, but hoping it's closely enough related that >>> I'll >>> be forgiven. :) >>> >>> If anyone has time/interest to try out FreeNAS 9.10 (based on FreeBSD >>> 10.3) - with Mellanox Infiniband card drivers baked in, there are two >>> ISO's here for testing: >>> >>> https://github.com/justinclift/freenas-infiniband​ >>> >>> One ISO is for native Infiniband mode (forcing IPoIB mode for the >>> cards): >>> >>> https://github.com/justinclift/freenas-infiniband/releases/tag/FreeNAS-IB-9.10-alpha1 >>> >>> The other ISO is for native Ethernet mode: >>> >>> https://github.com/justinclift/freenas-infiniband/releases/tag/FreeNAS-IB-EN-9.10-alpha1v10 >>> >>> InfiniHost through to ConnectX-3 cards should all work for the IB mode >>> one. ConnectX through ConnectX-3 should work for the Ethernet mode >>> one. >>> >>> And yeah, it would be better to have just one ISO. There is a weird >>> kernel panic bug I'm trying to track down first (triggered when >>> switched >>> modes), that needs fixing before that can happen. >>> >>> All feedback is hugely encouraged and welcome. :) >>> >>> Regards and best wishes, >>> >>> Justin Clift >>> >>> -- >>> "My grandfather once told me that there are two kinds of people: those >>> who work and those who take the credit. He told me to try to be in the >>> first group; there was less competition there." >>> - Indira Gandhi >>> >>> _______________________________________________ >>> freebsd-infiniband@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-infiniband >>> To unsubscribe, send any mail to >>> "freebsd-infiniband-unsubscribe@freebsd.org" > From owner-freebsd-infiniband@freebsd.org Fri Apr 29 21:21:11 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A9767B213CA for ; Fri, 29 Apr 2016 21:21:11 +0000 (UTC) (envelope-from justin@postgresql.org) Received: from meldrar.postgresql.org (meldrar.postgresql.org [IPv6:2a02:c0:301:0:ffff::31]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "*.postgresql.org", Issuer "Gandi Standard SSL CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6E1D81D83 for ; Fri, 29 Apr 2016 21:21:11 +0000 (UTC) (envelope-from justin@postgresql.org) Received: from 82-69-92-196.dsl.in-addr.zen.co.uk ([82.69.92.196] helo=[172.16.1.14]) by meldrar.postgresql.org with esmtpsa (TLS1.0:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1awFqS-0007PN-Tr; Fri, 29 Apr 2016 21:21:08 +0000 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Initial ISO's ready for testing FreeNAS 9.10 with Mellanox cards From: Justin Clift In-Reply-To: <0567682d-eaa4-332c-d6e6-f267f113f089@gmail.com> Date: Fri, 29 Apr 2016 22:21:03 +0100 Cc: freebsd-infiniband@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <65B3A04E-53D0-49CB-8DE0-FDE0D8F66B26@postgresql.org> <571E2BA5.5080500@gmail.com> <157d89e5f246d88cb08fa43f2fbf690e@postgresql.org> <0567682d-eaa4-332c-d6e6-f267f113f089@gmail.com> To: Jason Bacon X-Mailer: Apple Mail (2.1878.6) X-Pg-Spam-Score: -2.9 (--) X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Apr 2016 21:21:11 -0000 Ahhhhh. =46rom memory, the IPoIB ISO is using one of the earlier kernel = config's I knocked up... and ... er... I think I disabled a bunch of the disk = device controllers in that. Oops. *facepalm* :( I'll make a point to re-enable things for the next ISO (likely = tomorrow). I've just been putting time into writing up docs for a different project = today & yesterday, thus the lack of recent update. + Justin On 29 Apr 2016, at 22:09, Jason Bacon wrote: > Unfortunately, the installer (ipoib image) is failing on our PowerEdge = R730XD. >=20 > Abbreviated output below: >=20 > active set on mfid0 > No handlers could be found for logger "freenaOS.Configuration" > Traceback > freenas-install, line 56 > if installer.GetPackages() !=3D True > Instaler.py, line 857 > pkgFile =3D self._conf.FindPackageFile(pkg, = handler=3Dget_file_handler) > Configuration.py, line 1109 > intr_ok =3D True > Configuration.py, line 638 > raise url_exc > urllib2.URLError: urlopen error errno 8 hostname nor servname = provided, or not known > installation on mfid0 has failed >=20 > This happens about 10 seconds after selecting a boot disk and hitting = OK. It didn't get to network configuration or anything else, so the = urlopen error seems odd. >=20 > Note, however, that I was also unable to install FreeBSD with UFS on = this system due to the use of the MFI RAID driver. FreeBSD is on the = cusp of moving from MFI to MRSAS, so very new LSI controllers aren't = supported by MFI, which is still the default. If I install with ZFS, it = manages to limp through the installation, but RAID performance is = terrible with the MFI driver. After switching to MRSAS, performance is = very good. >=20 > Cheers, >=20 > Jason >=20 > On 04/25/16 09:46, Justin Clift wrote: >> Awesome, please do. :) >>=20 >> Noticed your post on the FreeNAS forum about this as well, so I've = posted info there too. The post is in moderation though, so hopefully = someone will approve it and others will test it too. :) >>=20 >> As a side note, I've been able to do basic interoperability testing = of the Ethernet mode ISO FreeNAS build with a Windows client (also = running ConnectX card). Trying the same thing in IPoIB mode isn't = working though. Seems like driver problems on the Windows side. :( >>=20 >> Currently seeking assistance with that through the Mellanox Community = forum (don't laugh... I'm not a Windows guy! :>). It's probably just = something simple. Hopefully. ;) >>=20 >> + Justin >>=20 >>=20 >> On 2016-04-25 15:37, Jason Bacon wrote: >>> Justin, >>>=20 >>> This is *exactly* what I'm interested in doing right now. In fact, = I >>> had been exploring how to do a FreeNAS build with IB support. Thanks >>> a million for providing the ISOs. >>>=20 >>> I can test with Infinihost and ConnectX-3. >>>=20 >>> Jason >>>=20 >>> On 04/25/16 09:03, Justin Clift wrote: >>>> This is semi-off-topic, but hoping it's closely enough related that = I'll >>>> be forgiven. :) >>>>=20 >>>> If anyone has time/interest to try out FreeNAS 9.10 (based on = FreeBSD >>>> 10.3) - with Mellanox Infiniband card drivers baked in, there are = two >>>> ISO's here for testing: >>>>=20 >>>> https://github.com/justinclift/freenas-infiniband=E2=80=8B >>>>=20 >>>> One ISO is for native Infiniband mode (forcing IPoIB mode for the = cards): >>>>=20 >>>> = https://github.com/justinclift/freenas-infiniband/releases/tag/FreeNAS-IB-= 9.10-alpha1 >>>>=20 >>>> The other ISO is for native Ethernet mode: >>>>=20 >>>> = https://github.com/justinclift/freenas-infiniband/releases/tag/FreeNAS-IB-= EN-9.10-alpha1v10 >>>>=20 >>>> InfiniHost through to ConnectX-3 cards should all work for the IB = mode >>>> one. ConnectX through ConnectX-3 should work for the Ethernet mode = one. >>>>=20 >>>> And yeah, it would be better to have just one ISO. There is a = weird >>>> kernel panic bug I'm trying to track down first (triggered when = switched >>>> modes), that needs fixing before that can happen. >>>>=20 >>>> All feedback is hugely encouraged and welcome. :) >>>>=20 >>>> Regards and best wishes, >>>>=20 >>>> Justin Clift >>>>=20 >>>> --=20 >>>> "My grandfather once told me that there are two kinds of people: = those >>>> who work and those who take the credit. He told me to try to be in = the >>>> first group; there was less competition there." >>>> - Indira Gandhi >>>>=20 >>>> _______________________________________________ >>>> freebsd-infiniband@freebsd.org mailing list >>>> https://lists.freebsd.org/mailman/listinfo/freebsd-infiniband >>>> To unsubscribe, send any mail to = "freebsd-infiniband-unsubscribe@freebsd.org" >>=20 >=20 -- "My grandfather once told me that there are two kinds of people: those who work and those who take the credit. He told me to try to be in the first group; there was less competition there." - Indira Gandhi From owner-freebsd-infiniband@freebsd.org Sat Apr 30 07:54:30 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C1FE6AD9EB7 for ; Sat, 30 Apr 2016 07:54:30 +0000 (UTC) (envelope-from ferrao@vmio.com.br) Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1bon0145.outbound.protection.outlook.com [157.56.111.145]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1AEF8132C for ; Sat, 30 Apr 2016 07:54:29 +0000 (UTC) (envelope-from ferrao@vmio.com.br) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vmio.onmicrosoft.com; s=selector1-vmio-com-br; h=From:To:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=6UxJGXVrSfapepd0psIbNoDg5OYTn19DIqaKu4/8brU=; b=M82zc5SENfYpBd3dXFJKBgqiEiMDSUlEHA0RqKbYHiV8zjPN4FEqPxfxGDY+WV1KPlN7w77HP6+Ly11YJrZt9WnJ/qFMPsQl6Znk1ZPPDbfKV+faYgSp0uOO7TMoxJWhzmGyka6bBzUJpMf2E2iyNKbqqbSAu4feZChNPST/bfg= Received: from RO1PR80MB1656.lamprd80.prod.outlook.com (10.165.142.154) by RO1PR80MB1653.lamprd80.prod.outlook.com (10.165.142.151) with Microsoft SMTP Server (TLS) id 15.1.485.9; Sat, 30 Apr 2016 07:54:18 +0000 Received: from RO1PR80MB1656.lamprd80.prod.outlook.com ([10.165.142.154]) by RO1PR80MB1656.lamprd80.prod.outlook.com ([10.165.142.154]) with mapi id 15.01.0485.011; Sat, 30 Apr 2016 07:54:18 +0000 From: =?utf-8?B?Vmluw61jaXVzIEZlcnLDo28=?= To: "freebsd-infiniband@freebsd.org" Subject: QLogic IBA7322 QDR InfiniBand HCA on 10.3-RELEASE Thread-Topic: QLogic IBA7322 QDR InfiniBand HCA on 10.3-RELEASE Thread-Index: AQHRorV+yFyqJvLPmE2rif2vRi2s9g== Date: Sat, 30 Apr 2016 07:54:16 +0000 Message-ID: <1E132EAE-8287-4407-BBBD-25319FB0091C@vmio.com.br> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: freebsd.org; dkim=none (message not signed) header.d=none;freebsd.org; dmarc=none action=none header.from=vmio.com.br; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [189.60.233.109] x-ms-office365-filtering-correlation-id: 8ec95a48-8e64-4a3b-43c2-08d370cca1fa x-microsoft-exchange-diagnostics: 1; RO1PR80MB1653; 5:BcSv9Rr2r/SzM5YHuzor2tEH/sQ88aPJ9A+EIUiBq+xuiJ7MO6IHBx8VLwdHbjOs+1P9kFIpAiVzm8vR0/2CbZZBeyk1ZI+HQB4EJ45meszkoWeSQDED5r2GlMHqQLxcF4svI9jP9kGUELwyeJMefw==; 24:87LDfd/dZHoZmSSRvl9xTvCKNGMTVWH8M1D74bpnNQr1LU6YhxkBSAX8uLJi/BME61e2PRyu3M9x5bR4hqh5443EiLEpVfT8oqyeWZeZs1M=; 7:n08IdyaT8LYrryFzerni7SWlepGohYWQubXD+OTTQASxtXXeRWXUfAyAsBUFGplCGKM0K78AmgiGlygxsfbHY7DlhjRuufR2J6w/eVKpHQvHWdFeBFuiTIEt2zvXcVwcnQmref+4AUx1YHfbSo0ym+kdSOoCVMLu50MZZkOsbu/8u313B00NuOiJm71xbsfr x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:RO1PR80MB1653; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(9101521072)(6040130)(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046)(6041072)(6043046)(6042046); SRVR:RO1PR80MB1653; BCL:0; PCL:0; RULEID:; SRVR:RO1PR80MB1653; x-forefront-prvs: 0928072091 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(51874003)(81166005)(2501003)(2906002)(189998001)(5640700001)(54356999)(50986999)(575784001)(86362001)(66066001)(36756003)(74482002)(2900100001)(3280700002)(122556002)(11100500001)(16236675004)(3660700001)(5004730100002)(77096005)(450100001)(5002640100001)(83716003)(107886002)(110136002)(15975445007)(33656002)(106116001)(1220700001)(82746002)(5008740100001)(2351001)(229853001)(19617315012)(87936001)(19580395003)(19580405001)(85182001)(10400500002)(6116002)(1096002)(85202003)(586003)(92566002)(3846002)(102836003)(11686003)(104396002); DIR:OUT; SFP:1102; SCL:1; SRVR:RO1PR80MB1653; H:RO1PR80MB1656.lamprd80.prod.outlook.com; FPR:; SPF:None; MLV:sfv; LANG:en; spamdiagnosticoutput: 1:23 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: vmio.com.br X-MS-Exchange-CrossTenant-originalarrivaltime: 30 Apr 2016 07:54:16.6293 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 3da02919-4b9d-42af-a7f7-a5975700d7c4 X-MS-Exchange-Transport-CrossTenantHeadersStamped: RO1PR80MB1653 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Apr 2016 07:54:30 -0000 SGVsbG8gZ3V5cywNCg0KSeKAmW0gcGxheWluZyB3aXRoIEluaWZpbmJhbmQgZm9yIHRoZSBmaXJz dCB0aW1lIG9uIEZyZWVCU0QgYW5kIEkgYWxyZWFkeSBoYXZlIHByb2JsZW1zIHdpdGggaXQuIEni gJl2ZSBnb3QgYSBtYWNoaW5lIHdpdGggdGhpcyBjYXJkIGluIHNwZWNpZmljIGFuZCBJ4oCZbSB0 cnlpbmcgdG8gdXNlIHRoaXMgY2FyZCB3aXRob3V0IGFueSBzdWNjZXNzLg0KDQpwY2ljb25mIC1s diBvdXRwdXRzIHRoaXM6DQpyb290QGZyZWVic2QxOn4gIyBwY2ljb25mIC1sdiB8IGdyZXAgLWkg UWxvZ2ljIC1BMyAtQjENCm5vbmUxN0BwY2kwOjI6MDowOiBjbGFzcz0weDBjMDYwMCBjYXJkPTB4 NzMyMjEwNzcgY2hpcD0weDczMjIxMDc3IHJldj0weDAyIGhkcj0weDAwDQogICAgdmVuZG9yICAg ICA9ICdRTG9naWMgQ29ycC4nDQogICAgZGV2aWNlICAgICA9ICdJQkE3MzIyIFFEUiBJbmZpbmlC YW5kIEhDQScNCiAgICBjbGFzcyAgICAgID0gc2VyaWFsIGJ1cw0KDQpTbyBhdCBsZWFzdCB0aGUg c3lzdGVtIGlzIGF3YXJlIG9mIHRoZSBjYXJkLCBidXQgSSBjYW7igJl0IHNlZSBpdCBvbiBkbWVz Zywgc28gc29tZXRoaW5nIGlzIG1pc3NpbmcuIERyaXZlcnM/IFN1cHBvcnQ/IEtlcm5lbCBNb2R1 bGU/DQoNCkkgY2FtZSBhY3Jvc3MgdGhpcyB3ZWJwYWdlIGh0dHBzOi8vd2lraS5mcmVlYnNkLm9y Zy9JbmZpbmlCYW5kIGFuZCBjYW5ub3QgZmluZCB0aGUgY2FyZCBvbiB0aGUgdGFibGUgb2Ygc3Vw cG9ydGVkIGhhcmR3YXJlLCBzbyBhdCB0aGlzIHBvaW50IEkgc3RhcnRlZCB0byBiZWxpZXZlIHRo YXQgdGhlIGNhcmQgaXMganVzdCB1bnN1cHBvcnRlZC4NCg0KQnV0IGhlcmUgd2UgYXJlIG9uIHRo ZSBtYWlsaW5nIGxpc3QsIHRvIHNlZSBpZiBhbnlvbmUgY2FuIGhlbHAuDQoNClRoYW5rcyBpbiBh ZHZhbmNlLg0KDQo= From owner-freebsd-infiniband@freebsd.org Sat Apr 30 09:03:27 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 466A5B0F903 for ; Sat, 30 Apr 2016 09:03:27 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (heidi.turbocat.net [88.198.202.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 107AA12C7 for ; Sat, 30 Apr 2016 09:03:26 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 5F2601FE022; Sat, 30 Apr 2016 11:03:17 +0200 (CEST) Subject: Re: QLogic IBA7322 QDR InfiniBand HCA on 10.3-RELEASE To: =?UTF-8?B?Vmluw61jaXVzIEZlcnLDo28=?= , "freebsd-infiniband@freebsd.org" References: <1E132EAE-8287-4407-BBBD-25319FB0091C@vmio.com.br> From: Hans Petter Selasky Message-ID: <5724759A.1090805@selasky.org> Date: Sat, 30 Apr 2016 11:06:34 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.5.0 MIME-Version: 1.0 In-Reply-To: <1E132EAE-8287-4407-BBBD-25319FB0091C@vmio.com.br> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Apr 2016 09:03:27 -0000 On 04/30/16 09:54, Vinícius Ferrão wrote: > Hello guys, > > I’m playing with Inifinband for the first time on FreeBSD and I already have problems with it. I’ve got a machine with this card in specific and I’m trying to use this card without any success. > > pciconf -lv outputs this: > root@freebsd1:~ # pciconf -lv | grep -i Qlogic -A3 -B1 > none17@pci0:2:0:0: class=0x0c0600 card=0x73221077 chip=0x73221077 rev=0x02 hdr=0x00 > vendor = 'QLogic Corp.' > device = 'IBA7322 QDR InfiniBand HCA' > class = serial bus > > So at least the system is aware of the card, but I can’t see it on dmesg, so something is missing. Drivers? Support? Kernel Module? > > I came across this webpage https://wiki.freebsd.org/InfiniBand and cannot find the card on the table of supported hardware, so at this point I started to believe that the card is just unsupported. > > But here we are on the mailing list, to see if anyone can help. > > Thanks in advance. Hi, Try loading qlxgb or qlxgbe. --HPS From owner-freebsd-infiniband@freebsd.org Sat Apr 30 10:56:45 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9BC3AAD9CA8 for ; Sat, 30 Apr 2016 10:56:45 +0000 (UTC) (envelope-from justin@postgresql.org) Received: from meldrar.postgresql.org (meldrar.postgresql.org [IPv6:2a02:c0:301:0:ffff::31]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "*.postgresql.org", Issuer "Gandi Standard SSL CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6110413E6 for ; Sat, 30 Apr 2016 10:56:45 +0000 (UTC) (envelope-from justin@postgresql.org) Received: from 82-69-92-196.dsl.in-addr.zen.co.uk ([82.69.92.196] helo=[172.16.1.14]) by meldrar.postgresql.org with esmtpsa (TLS1.0:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1awSZh-0001WY-6m; Sat, 30 Apr 2016 10:56:40 +0000 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: QLogic IBA7322 QDR InfiniBand HCA on 10.3-RELEASE From: Justin Clift In-Reply-To: <5724759A.1090805@selasky.org> Date: Sat, 30 Apr 2016 11:56:35 +0100 Cc: "freebsd-infiniband@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: References: <1E132EAE-8287-4407-BBBD-25319FB0091C@vmio.com.br> <5724759A.1090805@selasky.org> To: Hans Petter Selasky , =?windows-1252?Q?Vin=EDcius_Ferr=E3o?= X-Mailer: Apple Mail (2.1878.6) X-Pg-Spam-Score: -2.9 (--) X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Apr 2016 10:56:45 -0000 On 30 Apr 2016, at 10:06, Hans Petter Selasky wrote: > On 04/30/16 09:54, Vin=EDcius Ferr=E3o wrote: >> Hello guys, >>=20 >> I=92m playing with Inifinband for the first time on FreeBSD and I = already have problems with it. I=92ve got a machine with this card in = specific and I=92m trying to use this card without any success. >>=20 >> pciconf -lv outputs this: >> root@freebsd1:~ # pciconf -lv | grep -i Qlogic -A3 -B1 >> none17@pci0:2:0:0: class=3D0x0c0600 card=3D0x73221077 chip=3D0x73221077= rev=3D0x02 hdr=3D0x00 >> vendor =3D 'QLogic Corp.' >> device =3D 'IBA7322 QDR InfiniBand HCA' >> class =3D serial bus >>=20 >> So at least the system is aware of the card, but I can=92t see it on = dmesg, so something is missing. Drivers? Support? Kernel Module? >>=20 >> I came across this webpage https://wiki.freebsd.org/InfiniBand and = cannot find the card on the table of supported hardware, so at this = point I started to believe that the card is just unsupported. >>=20 >> But here we are on the mailing list, to see if anyone can help. >>=20 >> Thanks in advance. >=20 > Hi, >=20 > Try loading qlxgb or qlxgbe. Hmmm, between them, the man pages for qlxgb and qlxgbe say they support: * QLogic 3200 series (qlxgb) * QLogic 8200 series (qlxgb) * QLogic 8300 series (qlxgbe) Should we add them to the "hardware support by release" table? Vin=EDcius, with your card, do you know the exact model number? IBA7322 seems to be the controller chip on the card itself, not the adapter = model number, so it's hard to tell which series it belong too. Saying that from looking through the QLogic IB Adapters PDF: = http://filedownloads.qlogic.com/files/driver/78254/Install_Guide_QLogic_IB= _Adapters_A.pdf Page 3-5 mentions it, as well as Figure 3-1 on that same page. :) Regards and best wishes, Justin Clift -- "My grandfather once told me that there are two kinds of people: those who work and those who take the credit. He told me to try to be in the first group; there was less competition there." - Indira Gandhi From owner-freebsd-infiniband@freebsd.org Sat Apr 30 11:35:04 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1F62BAEE69E for ; Sat, 30 Apr 2016 11:35:04 +0000 (UTC) (envelope-from bacon4000@gmail.com) Received: from mail-io0-x22e.google.com (mail-io0-x22e.google.com [IPv6:2607:f8b0:4001:c06::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DB2C7142A for ; Sat, 30 Apr 2016 11:35:03 +0000 (UTC) (envelope-from bacon4000@gmail.com) Received: by mail-io0-x22e.google.com with SMTP id u185so152082150iod.3 for ; Sat, 30 Apr 2016 04:35:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=oXYAHCUZ2XohfseQAhK1DBs3TO45BN/Bq4y63K+BP2k=; b=Hgg1eulpje6kFAStQAaVhxTjL3ML0Q2ix4LDBmq3YVtgg4SAGDj3wC/5Adsq56u2c2 rsr0wvX8PMPZCYq1vmQu/ra9uKJcRiWGGi6/Ue96iV8YbedTw+AM9+5Ep1TdR4+c/diN a0sLFHBevf2uZoPhAVSNJSAlEkIJg3MhAgO2pSxQcDGN58fOdilxPkSgT/k6qn4MdTUP XcZyxEU3afZXuhjxGrrF0j+kaJ88iE8hhAq6uj50DwqsB/TrjcrX85v78qAwME0/BqwI ywmgMu3grdo6y/e0QdVqu8zpytTOeBa+akv/l663A2osqoynY9M06BQ62nl+DmzMd58h aAjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=oXYAHCUZ2XohfseQAhK1DBs3TO45BN/Bq4y63K+BP2k=; b=W52+2EqSSjdN2bgEBzrrHesXOdbotCl3pYsg0/nOyQeIqHgyZbdo/kL9lzzh7tkJ0g 9rNurd0HnkI/5rMYba27YhbIbXkMw0emA5xVM0OY6eLlnXtX4MU8A92h4djhoU/bJ7X6 o9ffIUU+m2Ol2y8mZ2bCADkIF9s1PPUjiR8MDRR4bwSGUwSRGqtcSUZnyrTMHDsrZjoa zxTXIc+aazWOdKv9uKZ03CQrHS3J9lQeSqhiHSu4Yb0yF1Em8DzL4TKjsuLUE6jMom8L r2/MDPMuXhG9Y/+14Y4gH2uSarm5vhiA4Zu7JzxH/OyfM9G/ichwtQpQA3CZu7Onx+dz zvMw== X-Gm-Message-State: AOPr4FXfirKNu2c7m76ElL1LRkNWEfypqEMfQj0x5wqA27QFcC8FCQwk/IHIgl97sdtcKg== X-Received: by 10.107.132.194 with SMTP id o63mr27893995ioi.118.1462016103364; Sat, 30 Apr 2016 04:35:03 -0700 (PDT) Received: from imacbsd.acadix.biz ([64.83.177.9]) by smtp.gmail.com with ESMTPSA id 33sm449916iom.17.2016.04.30.04.35.02 (version=TLSv1/SSLv3 cipher=OTHER); Sat, 30 Apr 2016 04:35:02 -0700 (PDT) Subject: Re: QLogic IBA7322 QDR InfiniBand HCA on 10.3-RELEASE To: Justin Clift , Hans Petter Selasky , =?UTF-8?B?Vmluw61jaXVzIEZlcnLDo28=?= References: <1E132EAE-8287-4407-BBBD-25319FB0091C@vmio.com.br> <5724759A.1090805@selasky.org> Cc: "freebsd-infiniband@freebsd.org" From: Jason Bacon Message-ID: <57249865.9060301@gmail.com> Date: Sat, 30 Apr 2016 06:35:01 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.7.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Apr 2016 11:35:04 -0000 On 04/30/16 05:56, Justin Clift wrote: > On 30 Apr 2016, at 10:06, Hans Petter Selasky wrote: >> On 04/30/16 09:54, Vinícius Ferrão wrote: >>> Hello guys, >>> >>> I’m playing with Inifinband for the first time on FreeBSD and I already have problems with it. I’ve got a machine with this card in specific and I’m trying to use this card without any success. >>> >>> pciconf -lv outputs this: >>> root@freebsd1:~ # pciconf -lv | grep -i Qlogic -A3 -B1 >>> none17@pci0:2:0:0: class=0x0c0600 card=0x73221077 chip=0x73221077 rev=0x02 hdr=0x00 >>> vendor = 'QLogic Corp.' >>> device = 'IBA7322 QDR InfiniBand HCA' >>> class = serial bus >>> >>> So at least the system is aware of the card, but I can’t see it on dmesg, so something is missing. Drivers? Support? Kernel Module? >>> >>> I came across this webpage https://wiki.freebsd.org/InfiniBand and cannot find the card on the table of supported hardware, so at this point I started to believe that the card is just unsupported. >>> >>> But here we are on the mailing list, to see if anyone can help. >>> >>> Thanks in advance. >> Hi, >> >> Try loading qlxgb or qlxgbe. > Hmmm, between them, the man pages for qlxgb and qlxgbe say they support: > > * QLogic 3200 series (qlxgb) > * QLogic 8200 series (qlxgb) > * QLogic 8300 series (qlxgbe) > > Should we add them to the "hardware support by release" table? > > Vinícius, with your card, do you know the exact model number? IBA7322 > seems to be the controller chip on the card itself, not the adapter model > number, so it's hard to tell which series it belong too. > > Saying that from looking through the QLogic IB Adapters PDF: > > http://filedownloads.qlogic.com/files/driver/78254/Install_Guide_QLogic_IB_Adapters_A.pdf > > Page 3-5 mentions it, as well as Figure 3-1 on that same page. :) > > Regards and best wishes, > > Justin Clift > > -- > "My grandfather once told me that there are two kinds of people: those > who work and those who take the credit. He told me to try to be in the > first group; there was less competition there." > - Indira Gandhi > > _______________________________________________ > freebsd-infiniband@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-infiniband > To unsubscribe, send any mail to "freebsd-infiniband-unsubscribe@freebsd.org" I'm not seeing any mention of Infiniband support in the 3200 or 8x00 series cards. They look to be strictly 10 GBE. Cheers, Jason -- All wars are civil wars, because all men are brothers ... Each one owes infinitely more to the human race than to the particular country in which he was born. -- Francois Fenelon From owner-freebsd-infiniband@freebsd.org Sat Apr 30 12:52:15 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 033C6B1CE23 for ; Sat, 30 Apr 2016 12:52:15 +0000 (UTC) (envelope-from justin@postgresql.org) Received: from meldrar.postgresql.org (meldrar.postgresql.org [IPv6:2a02:c0:301:0:ffff::31]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "*.postgresql.org", Issuer "Gandi Standard SSL CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B3EB814E7 for ; Sat, 30 Apr 2016 12:52:14 +0000 (UTC) (envelope-from justin@postgresql.org) Received: from 82-69-92-196.dsl.in-addr.zen.co.uk ([82.69.92.196] helo=[172.16.1.14]) by meldrar.postgresql.org with esmtpsa (TLS1.0:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1awUNU-0004J9-95; Sat, 30 Apr 2016 12:52:11 +0000 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: QLogic IBA7322 QDR InfiniBand HCA on 10.3-RELEASE From: Justin Clift In-Reply-To: <57249865.9060301@gmail.com> Date: Sat, 30 Apr 2016 13:52:05 +0100 Cc: Hans Petter Selasky , =?windows-1252?Q?Vin=EDcius_Ferr=E3o?= , "freebsd-infiniband@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: <001FE6A2-112B-40B2-BC38-C19A57012EA7@postgresql.org> References: <1E132EAE-8287-4407-BBBD-25319FB0091C@vmio.com.br> <5724759A.1090805@selasky.org> <57249865.9060301@gmail.com> To: Jason Bacon X-Mailer: Apple Mail (2.1878.6) X-Pg-Spam-Score: -2.9 (--) X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Apr 2016 12:52:15 -0000 On 30 Apr 2016, at 12:35, Jason Bacon wrote: > On 04/30/16 05:56, Justin Clift wrote: >> On 30 Apr 2016, at 10:06, Hans Petter Selasky = wrote: >>> On 04/30/16 09:54, Vin=EDcius Ferr=E3o wrote: >>>> Hello guys, >>>>=20 >>>> I=92m playing with Inifinband for the first time on FreeBSD and I = already have problems with it. I=92ve got a machine with this card in = specific and I=92m trying to use this card without any success. >>>>=20 >>>> pciconf -lv outputs this: >>>> root@freebsd1:~ # pciconf -lv | grep -i Qlogic -A3 -B1 >>>> none17@pci0:2:0:0: class=3D0x0c0600 card=3D0x73221077 = chip=3D0x73221077 rev=3D0x02 hdr=3D0x00 >>>> vendor =3D 'QLogic Corp.' >>>> device =3D 'IBA7322 QDR InfiniBand HCA' >>>> class =3D serial bus >>>>=20 >>>> So at least the system is aware of the card, but I can=92t see it = on dmesg, so something is missing. Drivers? Support? Kernel Module? >>>>=20 >>>> I came across this webpage https://wiki.freebsd.org/InfiniBand and = cannot find the card on the table of supported hardware, so at this = point I started to believe that the card is just unsupported. >>>>=20 >>>> But here we are on the mailing list, to see if anyone can help. >>>>=20 >>>> Thanks in advance. >>> Hi, >>>=20 >>> Try loading qlxgb or qlxgbe. >> Hmmm, between them, the man pages for qlxgb and qlxgbe say they = support: >>=20 >> * QLogic 3200 series (qlxgb) >> * QLogic 8200 series (qlxgb) >> * QLogic 8300 series (qlxgbe) >>=20 >> Should we add them to the "hardware support by release" table? >>=20 >> Vin=EDcius, with your card, do you know the exact model number? = IBA7322 >> seems to be the controller chip on the card itself, not the adapter = model >> number, so it's hard to tell which series it belong too. >>=20 >> Saying that from looking through the QLogic IB Adapters PDF: >>=20 >> = http://filedownloads.qlogic.com/files/driver/78254/Install_Guide_QLogic_IB= _Adapters_A.pdf >>=20 >> Page 3-5 mentions it, as well as Figure 3-1 on that same page. :) >=20 > I'm not seeing any mention of Infiniband support in the 3200 or 8x00 = series cards. They look to be strictly 10 GBE. Ahhh, that explains a few things then. :) + Justin -- "My grandfather once told me that there are two kinds of people: those who work and those who take the credit. He told me to try to be in the first group; there was less competition there." - Indira Gandhi