From owner-freebsd-infiniband@freebsd.org Tue Aug 16 20:23:53 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 277E1BBCCD6 for ; Tue, 16 Aug 2016 20:23:53 +0000 (UTC) (envelope-from alex.bowden@outlook.com) Received: from COL004-OMC3S7.hotmail.com (col004-omc3s7.hotmail.com [65.55.34.145]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "*.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DBC931F42 for ; Tue, 16 Aug 2016 20:23:52 +0000 (UTC) (envelope-from alex.bowden@outlook.com) Received: from NAM01-BY2-obe.outbound.protection.outlook.com ([65.55.34.135]) by COL004-OMC3S7.hotmail.com over TLS secured channel with Microsoft SMTPSVC(7.5.7601.23008); Tue, 16 Aug 2016 13:23:46 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=ls31ROxlb1M+wZIRB8k1TzTjIqwe9MCqn/CWJI0UI0c=; b=omTB4BvVF/z8IF+gY1kdOkIq+26jYzNXAoDC3MBuDqdPumEogTucsvGj843pLq7YzuTLNV+2Y0ivX4Ai9H7tuTfHqUw6XA3KekqQ9LsrEqThZRTAqy4ZBNXuJBTO2L2WthO+daYRFYoErNqP6WbnyJW/811AQSHnQhWPrlh3cbxrWnSIoqh4rWGGfZtaVbYT1uBA8pNqGNQJ9DzsQg/DuTGbNxQN2xFlwF04mNdm5Lza3jG2k/+a8Wg5ZAFcHOobXR71b8+IEYywb0FvTYvE/QXn7Pg8NJZWuYFEXjjZSE9OtFpUPLC/jDenoem69F7JTzIwtrlauSHIlvaccuzPWA== Received: from BN3NAM01FT023.eop-nam01.prod.protection.outlook.com (10.152.66.53) by BN3NAM01HT195.eop-nam01.prod.protection.outlook.com (10.152.67.84) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384) id 15.1.567.7; Tue, 16 Aug 2016 20:23:45 +0000 Received: from DM2PR05MB751.namprd05.prod.outlook.com (10.152.66.60) by BN3NAM01FT023.mail.protection.outlook.com (10.152.67.155) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384) id 15.1.567.7 via Frontend Transport; Tue, 16 Aug 2016 20:23:45 +0000 Received: from DM2PR05MB751.namprd05.prod.outlook.com ([10.141.178.148]) by DM2PR05MB751.namprd05.prod.outlook.com ([10.141.178.148]) with mapi id 15.01.0587.005; Tue, 16 Aug 2016 20:23:44 +0000 From: Alex Bowden To: "freebsd-infiniband@freebsd.org" Subject: Set non blocking on /dev/rdma_cm Thread-Topic: Set non blocking on /dev/rdma_cm Thread-Index: AQHR9/Ra19SP/34IP0+fVdY/IpOONw== Date: Tue, 16 Aug 2016 20:23:44 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=softfail (sender IP is 10.152.66.60) smtp.mailfrom=outlook.com; freebsd.org; dkim=none (message not signed) header.d=none;freebsd.org; dmarc=fail action=none header.from=outlook.com; received-spf: SoftFail (protection.outlook.com: domain of transitioning outlook.com discourages use of 10.152.66.60 as permitted sender) x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [skKhKoMi2g8LTIrsdlk2/2G6YWD6WQKB] x-eopattributedmessage: 0 x-forefront-antispam-report: CIP:10.152.66.60; IPV:NLI; CTRY:; EFV:NLI; SFV:NSPM; SFS:(10019020)(98900003); DIR:OUT; SFP:1102; SCL:1; SRVR:BN3NAM01HT195; H:DM2PR05MB751.namprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; x-microsoft-exchange-diagnostics: 1; BN3NAM01HT195; 6:tkNnGXdkhSYnUSnXGI+R5Ryiv33jjfLlKGxl0MdCFBFcQThg+TYORrzHf1E8e/c70UvVXoriFW/J1lf1uj+v8QWD/g964+u/KCWHLUY6qf6VOLzru7YAOTpdt7RxLV1lE1YBwhl5RmeWDrXeiFt/uK+301atdDmD1BHMrIEjbdkBmGzm4aCF/U2XSTibTXsGa1ACjShn5Xt8kHleNidgvevie5diemhU4owg9h2zC7H+8MMCMYv68UDrmCFA2veKW0MuBCeCR1WK3uT5FteethCIjQxmSnreffmnjSOKS9WeqosgDuJDGMpa4T/ikQFF; 5:1ztwOhHqyHZfe21H7UcIvzHe88DrIT3IRl6lJ2g8Fbru0azzAGNMrTnWLsBEm+niB3SdtyD51MuHQ4DxFcIWrq25sBufMBsMJgxdoC34oo9hJXb5N1WUGoar/nB69QXvXwzRQ1Iu4pWVs9ItUNwZ1A==; 24:tY44cNDiNq2EA2x81V/erDlpvIpP+lt3+KeOoOyQnc9UvxET/kVN31MwvXzO+iCrU+4hHfSQXxYCLkIHjF+yxgeit4PX7J99NgDUU5d6P1g=; 7:hgZACQCW0//Lqeo03TqnHjM0QIIR0z/sqOumhRccJceeCjgNgIWMVaeEXsebCymIThQ3uoDpQ54kB+cGTj8qGb/ccSEBSC+N/MVrE12qJ7ywgE8CGcGfSMxmvBhqs3QlZ6ml4CQp7s4z/sXs13k9hC3oleZdGTk9TsQnKsn9Xf//ZHl6R+YNJSpQRpeZPTPDs/x489MblvrD6CocuJb14f9HTddASkpUAv95bSJLK/5KAKBLSbf5cjV14BCapHRs x-ms-office365-filtering-correlation-id: 51b27935-22dd-4137-6f0a-08d3c6133864 x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(1601124038)(1601125047); SRVR:BN3NAM01HT195; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(432015012)(82015046); SRVR:BN3NAM01HT195; BCL:0; PCL:0; RULEID:; SRVR:BN3NAM01HT195; x-forefront-prvs: 0036736630 spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Aug 2016 20:23:44.1715 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Internet X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN3NAM01HT195 X-OriginalArrivalTime: 16 Aug 2016 20:23:46.0653 (UTC) FILETIME=[176158D0:01D1F7FC] Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Aug 2016 20:23:53 -0000 Hi all, I've modified rping to set O_NONBLOCKING on /dev/rdma_cm on the event chann= el (rdma_create_event_channel) which fails like this: 9997 rping CALL openat(AT_FDCWD,0x800a42708,0x100002,0) 9997 rping NAMI "/dev/rdma_cm" 9997 rping RET openat 3 ..snip.. 9997 rping CALL fcntl(0x3,F_SETFL,0x6) 9997 rping RET fcntl -1 errno 25 Inappropriate ioctl for device Is this not supported for some reason? This is with the latest OFED v3.2 pa= tches by Hans. The manpage for rdma_create_event_channel seems to state you can set non-bl= ocking on it. -Alex From owner-freebsd-infiniband@freebsd.org Tue Aug 16 20:32:08 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 033A4BBB00D for ; Tue, 16 Aug 2016 20:32:08 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (heidi.turbocat.net [88.198.202.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C32E71454 for ; Tue, 16 Aug 2016 20:32:07 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 3DC911FE022; Tue, 16 Aug 2016 22:32:05 +0200 (CEST) Subject: Re: Set non blocking on /dev/rdma_cm To: Alex Bowden , "freebsd-infiniband@freebsd.org" References: From: Hans Petter Selasky Message-ID: <3a373d74-9822-ed46-2788-fbc8d169a8a6@selasky.org> Date: Tue, 16 Aug 2016 22:36:33 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Aug 2016 20:32:08 -0000 On 08/16/16 22:23, Alex Bowden wrote: > Hi all, > > > I've modified rping to set O_NONBLOCKING on /dev/rdma_cm on the event channel (rdma_create_event_channel) which fails like this: > > > 9997 rping CALL openat(AT_FDCWD,0x800a42708,0x100002,0) > 9997 rping NAMI "/dev/rdma_cm" > 9997 rping RET openat 3 > ..snip.. > > 9997 rping CALL fcntl(0x3,F_SETFL,0x6) > 9997 rping RET fcntl -1 errno 25 Inappropriate ioctl for device > > Is this not supported for some reason? This is with the latest OFED v3.2 patches by Hans. Hi Alex, I'll check the code tomorrow. Quick answer: F_SETFL should be translated into FIONBIO and FASYNC ioctls in the FreeBSD kernel. Maybe there is a missing "case FIONBIO: return (0); /* success */" somewhere in the /dev/rdma_cm implementation, because the "fflags" handling is factored out, and only returning success on the IOCTL is required. --HPS From owner-freebsd-infiniband@freebsd.org Wed Aug 17 07:03:16 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 68E9FBBDE62 for ; Wed, 17 Aug 2016 07:03:16 +0000 (UTC) (envelope-from Krishna2@chelsio.com) Received: from NAM02-CY1-obe.outbound.protection.outlook.com (mail-cys01nam02on0125.outbound.protection.outlook.com [104.47.37.125]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0B6FB1E71 for ; Wed, 17 Aug 2016 07:03:15 +0000 (UTC) (envelope-from Krishna2@chelsio.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chelsious.onmicrosoft.com; s=selector1-chelsio-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=WaZkLWugl4FKB9jD2+OqM5uywecIUI7MzNip2vlDqTA=; b=wJnXA3OVs+76mf9YUIRvBewvj+O0MO9jebDoyZaIuwDou9kKE6uzVAh6FNC9Ahmu5ypZxvOqpDmwlfJk/bKVgcSBnj2nK3DnU3O+fBWkdCaqz3t28DI9g7uPuIGs+bI87Znj9rXED7uwYLNc0aJtDQtglqDh6MLvbDASEROoOps= Received: from BN4PR12MB0818.namprd12.prod.outlook.com (10.164.58.152) by BN4PR12MB0818.namprd12.prod.outlook.com (10.164.58.152) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384) id 15.1.587.9; Wed, 17 Aug 2016 07:03:07 +0000 Received: from BN4PR12MB0818.namprd12.prod.outlook.com ([10.164.58.152]) by BN4PR12MB0818.namprd12.prod.outlook.com ([10.164.58.152]) with mapi id 15.01.0587.009; Wed, 17 Aug 2016 07:03:07 +0000 From: KrishnamRaju ErapaRaju To: Alex Bowden , "freebsd-infiniband@freebsd.org" Subject: RE: Set non blocking on /dev/rdma_cm Thread-Topic: Set non blocking on /dev/rdma_cm Thread-Index: AQHR9/Ra19SP/34IP0+fVdY/IpOON6BMt7qw Date: Wed, 17 Aug 2016 07:03:07 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Krishna2@chelsio.com; x-originating-ip: [111.93.130.157] x-ms-office365-filtering-correlation-id: 9e32014e-f21b-455b-872f-08d3c66c8a7e x-microsoft-exchange-diagnostics: 1; BN4PR12MB0818; 6:E5I6nGMUz/Xfw1AEST4hk55NbiloMw3SbAQisaohC8X+olE/yoqGIm5i26q1CJoXBP/NSbOP7tsShsuzlmzpmWBjIFzNGZyw//5HtS0URyl0XA+qheRS285mChG5WgIMag91nECtYEsp8FBbQc4Bb1GRxrewS4XFUPRCsHM65c2sje9XSWsUC2Uh8U+uKIxkHO/SZvj+8agV2t3pTpqqR0ywN7cCkMZ6pwW2cs5zK+SMkrkL+kjiSinOoOwU51Ju0pMtr0Ol3UIyzZm02547zencpZ/ztB/XZj0I1xQk9btiuGFsCR6rW02LlfA1iNrp; 5:TjqyJ6mmQbHKTNRxatgCYM9t0JuHHO4bC9K1Tl5ftRws4VPH3S6D+hqavvJ0Rwj1aVAr8RHBVjZHpdM+MI8bozCajsUFw2efAnfH49M9k0kNzidm+iZuceQIoMu3/u/7G0cLNHEeLSo/66CvNPcTog==; 24:pUEPMZ516qzMRxoQ0UPir5Ft+kNRFzp6OHAR3c4e4+Oki3B9AASImFJrKfqrepoz3c564JAMn+1JCdl+4FUhrIfpYZlkE5jn7DmVjjfuAQA=; 7:uH3wlAG+/1dEjy6hZPvHe/5TjabSpiB/f3yV0eFzIEpMvzL4EdqXhpBxhDuUrsBcXoMGnz1zb94lEPEk9+YGm4OGH/GmZJ/l1ITqvjZ/PE8cfPXZ9CixMJoBkkjAEZ7bt41eLLn6HfFSTzUM/9RawgqoZ5xq4ArSoY0p8NVYaP1fFsj5vjuDACPuwLjYmbLxk/cfYdlGlBTa+YJ2BaySdwrpwbxFimVtqIX4oB3NY4RvP82N+QzfICa8aCbtZasK x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BN4PR12MB0818; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(75325880899374); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040176)(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046)(6043046)(6042046); SRVR:BN4PR12MB0818; BCL:0; PCL:0; RULEID:; SRVR:BN4PR12MB0818; x-forefront-prvs: 0037FD6480 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(6009001)(7916002)(199003)(189002)(13464003)(377454003)(53754006)(6116002)(3846002)(87936001)(33656002)(102836003)(122556002)(586003)(77096005)(2900100001)(15975445007)(2950100001)(8676002)(3660700001)(11100500001)(9686002)(8936002)(2501003)(76576001)(92566002)(2906002)(81156014)(3280700002)(106116001)(99286002)(105586002)(66066001)(19580395003)(19580405001)(10400500002)(305945005)(107886002)(50986999)(101416001)(189998001)(7846002)(7736002)(7696003)(106356001)(5002640100001)(54356999)(74316002)(97736004)(86362001)(76176999)(81166006)(68736007)(5001770100001); DIR:OUT; SFP:1102; SCL:1; SRVR:BN4PR12MB0818; H:BN4PR12MB0818.namprd12.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; received-spf: None (protection.outlook.com: chelsio.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: chelsio.com X-MS-Exchange-CrossTenant-originalarrivaltime: 17 Aug 2016 07:03:07.0116 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 065db76d-a7ae-4c60-b78a-501e8fc17095 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN4PR12MB0818 X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Aug 2016 07:03:16 -0000 Hi Alex,=20 I too faced the same issue earlier. But could not look into it/report it d= ue to other priorities. Hans, if you have root caused it please point me to the patch, few of my ap= ps uses O_NONBLOCK. Thanks, Krishna. -----Original Message----- From: owner-freebsd-infiniband@freebsd.org [mailto:owner-freebsd-infiniband= @freebsd.org] On Behalf Of Alex Bowden Sent: Wednesday, August 17, 2016 1:54 AM To: freebsd-infiniband@freebsd.org Subject: Set non blocking on /dev/rdma_cm Hi all, I've modified rping to set O_NONBLOCKING on /dev/rdma_cm on the event chann= el (rdma_create_event_channel) which fails like this: 9997 rping CALL openat(AT_FDCWD,0x800a42708,0x100002,0) 9997 rping NAMI "/dev/rdma_cm" 9997 rping RET openat 3 ..snip.. 9997 rping CALL fcntl(0x3,F_SETFL,0x6) 9997 rping RET fcntl -1 errno 25 Inappropriate ioctl for device Is this not supported for some reason? This is with the latest OFED v3.2 pa= tches by Hans. The manpage for rdma_create_event_channel seems to state you can set non-bl= ocking on it. -Alex _______________________________________________ freebsd-infiniband@freebsd.org mailing list https://lists.freebsd.org/mailm= an/listinfo/freebsd-infiniband To unsubscribe, send any mail to "freebsd-infiniband-unsubscribe@freebsd.or= g" From owner-freebsd-infiniband@freebsd.org Wed Aug 17 09:26:24 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 76483BBD074 for ; Wed, 17 Aug 2016 09:26:24 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (heidi.turbocat.net [88.198.202.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3C7911A3B for ; Wed, 17 Aug 2016 09:26:23 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 838BA1FE022; Wed, 17 Aug 2016 11:26:20 +0200 (CEST) Subject: Re: Set non blocking on /dev/rdma_cm To: KrishnamRaju ErapaRaju , Alex Bowden , "freebsd-infiniband@freebsd.org" References: From: Hans Petter Selasky Message-ID: <516f3566-33f0-c1d6-ab2d-997290a16ae5@selasky.org> Date: Wed, 17 Aug 2016 11:30:49 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/mixed; boundary="------------000922E94DAC204296CB83A8" X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Aug 2016 09:26:24 -0000 This is a multi-part message in MIME format. --------------000922E94DAC204296CB83A8 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit On 08/17/16 09:03, KrishnamRaju ErapaRaju wrote: > Hi Alex, > > > I too faced the same issue earlier. But could not look into it/report it due to other priorities. > > Hans, if you have root caused it please point me to the patch, few of my apps uses O_NONBLOCK. > Hi Alex, Can you try the attached patch? cd /usr/src cat ibcore_fionbio.diff | patch -p1 --HPS --------------000922E94DAC204296CB83A8 Content-Type: text/x-patch; name="ibcore_fionbio.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="ibcore_fionbio.diff" diff --git a/sys/ofed/drivers/infiniband/core/ucma.c b/sys/ofed/drivers/infiniband/core/ucma.c index 56f8786..2c87db1 100644 --- a/sys/ofed/drivers/infiniband/core/ucma.c +++ b/sys/ofed/drivers/infiniband/core/ucma.c @@ -49,6 +49,8 @@ #include #include +#include + /* RDMA Userspace Connection Manager Access */ static unsigned int max_backlog = 1024; @@ -1825,11 +1827,24 @@ static int ucma_close(struct inode *inode, struct file *filp) return 0; } +static long +ucma_ioctl(struct file *filp, unsigned int cmd, unsigned long arg) +{ + + switch (cmd) { + case FIONBIO: + return (0); + default: + return (-ENOTTY); + } +} + static const struct file_operations ucma_fops = { .owner = THIS_MODULE, .open = ucma_open, .release = ucma_close, .write = ucma_write, + .unlocked_ioctl = ucma_ioctl, .poll = ucma_poll, .llseek = no_llseek, }; --------------000922E94DAC204296CB83A8-- From owner-freebsd-infiniband@freebsd.org Wed Aug 17 09:31:15 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 961DEBBD1BA for ; Wed, 17 Aug 2016 09:31:15 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (heidi.turbocat.net [88.198.202.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5A3EB1C87 for ; Wed, 17 Aug 2016 09:31:14 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id ED2511FE023; Wed, 17 Aug 2016 11:31:12 +0200 (CEST) Subject: Re: Set non blocking on /dev/rdma_cm To: KrishnamRaju ErapaRaju , Alex Bowden , "freebsd-infiniband@freebsd.org" References: <516f3566-33f0-c1d6-ab2d-997290a16ae5@selasky.org> From: Hans Petter Selasky Message-ID: Date: Wed, 17 Aug 2016 11:35:42 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.0 MIME-Version: 1.0 In-Reply-To: <516f3566-33f0-c1d6-ab2d-997290a16ae5@selasky.org> Content-Type: multipart/mixed; boundary="------------9D6206EFA0B1F1CE84EBEBB7" X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Aug 2016 09:31:15 -0000 This is a multi-part message in MIME format. --------------9D6206EFA0B1F1CE84EBEBB7 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit On 08/17/16 11:30, Hans Petter Selasky wrote: > On 08/17/16 09:03, KrishnamRaju ErapaRaju wrote: >> Hi Alex, >> >> >> I too faced the same issue earlier. But could not look into it/report >> it due to other priorities. >> >> Hans, if you have root caused it please point me to the patch, few of >> my apps uses O_NONBLOCK. >> > > Hi Alex, > > Can you try the attached patch? > > cd /usr/src > cat ibcore_fionbio.diff | patch -p1 > > --HPS > Hi, Looks like we also need to support FIOASYNC. Updated patch attached. --HPS --------------9D6206EFA0B1F1CE84EBEBB7 Content-Type: text/x-patch; name="ibcore_fionbio.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="ibcore_fionbio.diff" diff --git a/sys/ofed/drivers/infiniband/core/ucma.c b/sys/ofed/drivers/infiniband/core/ucma.c index 56f8786..eb10aa8 100644 --- a/sys/ofed/drivers/infiniband/core/ucma.c +++ b/sys/ofed/drivers/infiniband/core/ucma.c @@ -49,6 +49,8 @@ #include #include +#include + /* RDMA Userspace Connection Manager Access */ static unsigned int max_backlog = 1024; @@ -1825,11 +1827,25 @@ static int ucma_close(struct inode *inode, struct file *filp) return 0; } +static long +ucma_ioctl(struct file *filp, unsigned int cmd, unsigned long arg) +{ + + switch (cmd) { + case FIONBIO: + case FIOASYNC: + return (0); + default: + return (-ENOTTY); + } +} + static const struct file_operations ucma_fops = { .owner = THIS_MODULE, .open = ucma_open, .release = ucma_close, .write = ucma_write, + .unlocked_ioctl = ucma_ioctl, .poll = ucma_poll, .llseek = no_llseek, }; --------------9D6206EFA0B1F1CE84EBEBB7-- From owner-freebsd-infiniband@freebsd.org Wed Aug 17 16:38:17 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BF9C0BBD05E for ; Wed, 17 Aug 2016 16:38:17 +0000 (UTC) (envelope-from alex.bowden@outlook.com) Received: from BAY004-OMC2S3.hotmail.com (bay004-omc2s3.hotmail.com [65.54.190.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "*.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 89DAC1A9D for ; Wed, 17 Aug 2016 16:38:17 +0000 (UTC) (envelope-from alex.bowden@outlook.com) Received: from NAM03-CO1-obe.outbound.protection.outlook.com ([65.54.190.124]) by BAY004-OMC2S3.hotmail.com over TLS secured channel with Microsoft SMTPSVC(7.5.7601.23008); Wed, 17 Aug 2016 09:38:11 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=0WQmSuEXrF//JXP0Ycazic3u+UfO21GUTHhdS8sStG0=; b=DoYvgujQD7xdsuB5Foihuk9feYHMwJztfyrfyl6GP6mvP+6n1ENt8gvocmUhLb32W/igWtYYcrbAvRR6kGJuOIXgBEob0DemRnB0/rhB7DdVWRGHL5Vit1FHtuy2GZ/oYgLgxwaUARr4+GSSsn8fYMlqPM11FO27bj+/4BgEpgndX7yWdsuwBicMFyymoYf/rBH/yGSqPx5HLGMo7loxihiOAt7T9ye7d+Wsl24Blofzjl6yCBBWyaHo7aBLcYWsKsppJ6htefGi5s4B31H9I83a0DUdis1+XyIKjaGiVqfMrZC1pCFR/YRs75urNStlMC7h+5ro859U5jQPKtn2Qg== Received: from BY2NAM03FT025.eop-NAM03.prod.protection.outlook.com (10.152.84.57) by BY2NAM03HT193.eop-NAM03.prod.protection.outlook.com (10.152.85.92) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384) id 15.1.587.6; Wed, 17 Aug 2016 16:38:10 +0000 Received: from DM2PR05MB751.namprd05.prod.outlook.com (10.152.84.55) by BY2NAM03FT025.mail.protection.outlook.com (10.152.84.232) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384) id 15.1.587.6 via Frontend Transport; Wed, 17 Aug 2016 16:38:09 +0000 Received: from DM2PR05MB751.namprd05.prod.outlook.com ([10.141.178.148]) by DM2PR05MB751.namprd05.prod.outlook.com ([10.141.178.148]) with mapi id 15.01.0587.005; Wed, 17 Aug 2016 16:38:08 +0000 From: Alex Bowden To: KrishnamRaju ErapaRaju , Hans Petter Selasky , "freebsd-infiniband@freebsd.org" Subject: Re: Set non blocking on /dev/rdma_cm Thread-Topic: Set non blocking on /dev/rdma_cm Thread-Index: AQHR9/Ra19SP/34IP0+fVdY/IpOON6BMt7qwgAAscYCAAAFdAIAAR52AgAAuNgE= Date: Wed, 17 Aug 2016 16:38:07 +0000 Message-ID: References: <516f3566-33f0-c1d6-ab2d-997290a16ae5@selasky.org> , In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=softfail (sender IP is 10.152.84.55) smtp.mailfrom=outlook.com; chelsio.com; dkim=none (message not signed) header.d=none;chelsio.com; dmarc=fail action=none header.from=outlook.com; received-spf: SoftFail (protection.outlook.com: domain of transitioning outlook.com discourages use of 10.152.84.55 as permitted sender) x-incomingtopheadermarker: TopHeader x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [UBpq/wvyM3qYVaR8MjX0/yuDvXWaijPZ] x-incomingheadercount: 38 x-eopattributedmessage: 0 x-forefront-antispam-report: CIP:10.152.84.55; IPV:NLI; CTRY:; EFV:NLI; SFV:NSPM; SFS:(10019020)(98900003); DIR:OUT; SFP:1102; SCL:1; SRVR:BY2NAM03HT193; H:DM2PR05MB751.namprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; x-microsoft-exchange-diagnostics: 1; BY2NAM03HT193; 6:CnLAkEtySoh0XQscKfqTRxSvZMinnLIZpVY4l3/omLCQKL76kK86vnhSvy5J9qtapffnIVhbdH6VNZDmJYjGrIhuvmxvuzVGzv9IhFy8XcRZ3pMfD9x3FORQIdu5buFVl3nS/luhVV0zV8aGHYFzSbLOExbFTwkNotTu1VabRL1devjsZKHdTz0ZU6tCHGbqnLsBdhWJr4AiKHLpm99SMjy12edaGO3GAp4R3bgGq2Unt3iDK1YI1EJPI8HkZpiylZlXVAp1upNPfMmoOdDSurg2bIaLCb1uLXMyCI0H+CSqQ6lMuLoqVQRL8D2MM5nd; 5:OEwlxEMOLJrxmkt5BTomx7ynekYc0ugIkbUVbBhPTYswYOgfuI0xRxqYfiwgD3PrsEd7BMZfuthNdtQi4TMETvCoxWH8m/h14wXGz/zyiKXsz2IOZc4UZdzyNBN8NNwa9xteeWlDXxv+EcRfpZwOTg==; 24:z0Ge0rxSWbiu+4o57J34NzXx8hHAMa3/WbIE68SEFFCU0CB3PsUeHvz8tlHu5O1gWXFfK9r/I9snCctiPiAsYS9oXWuXhcQoi/bQCVpPOCE=; 7:2SklMa38IVGfN47lI7IkKin0g/syiSpahFKNCO9AWvnQvPek7o0baYgvMx+co0nEtCCgezu1Nd+D7J3/Bg4j/Vfry5GvpqmbQ7MJIetDTrYboC4sfgNhhnPlX26Tnwo/hcWxOQ977cNVtMySKt3zAXG9z3K1swTnC4CLjjod+X1a/gZ38mtRLpTKPzqaSc9+EJmKasUvGGRK7ZQkCz3p2rf4PnFiU/GOYEw+ryEuEUcbH9WqQcwPsf+A/ShcHmXf x-ms-office365-filtering-correlation-id: 8c3aa7b4-5e36-4d0c-46d1-08d3c6bcdeb1 x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(1601124038)(1601125047); SRVR:BY2NAM03HT193; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(432015012)(82015046); SRVR:BY2NAM03HT193; BCL:0; PCL:0; RULEID:; SRVR:BY2NAM03HT193; x-forefront-prvs: 0037FD6480 spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-originalarrivaltime: 17 Aug 2016 16:38:07.9601 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Internet X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY2NAM03HT193 X-OriginalArrivalTime: 17 Aug 2016 16:38:11.0503 (UTC) FILETIME=[BE371BF0:01D1F8A5] Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Aug 2016 16:38:17 -0000 Hey Hans & Krishna, I've applied the patches. I've got past the initial fcntl call. Do we need = to add FIONWRITE and FIONREAD? -Alex ________________________________ From: KrishnamRaju ErapaRaju Sent: Wednesday, August 17, 2016 9:52 AM To: Hans Petter Selasky; Alex Bowden; freebsd-infiniband@freebsd.org Subject: RE: Set non blocking on /dev/rdma_cm Hi Hans, The error "Inappropriate ioctl for device" is not occurring after applying = your patch. Will check whether rdma_get_cm_event() is blocking or not when fcntl(O_NONB= LOCK) is set. Thanks, Krishna. -----Original Message----- From: Hans Petter Selasky [mailto:hps@selasky.org] Sent: Wednesday, August 17, 2016 3:06 PM To: KrishnamRaju ErapaRaju ; Alex Bowden ; freebsd-infiniband@freebsd.org Subject: Re: Set non blocking on /dev/rdma_cm On 08/17/16 11:30, Hans Petter Selasky wrote: > On 08/17/16 09:03, KrishnamRaju ErapaRaju wrote: >> Hi Alex, >> >> >> I too faced the same issue earlier. But could not look into >> it/report it due to other priorities. >> >> Hans, if you have root caused it please point me to the patch, few of >> my apps uses O_NONBLOCK. >> > > Hi Alex, > > Can you try the attached patch? > > cd /usr/src > cat ibcore_fionbio.diff | patch -p1 > > --HPS > Hi, Looks like we also need to support FIOASYNC. Updated patch attached. --HPS From owner-freebsd-infiniband@freebsd.org Wed Aug 17 16:48:03 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3473BBBD2B0 for ; Wed, 17 Aug 2016 16:48:03 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (mail.turbocat.net [IPv6:2a01:4f8:d16:4514::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F234B1EE2 for ; Wed, 17 Aug 2016 16:48:02 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 6B79C1FE022; Wed, 17 Aug 2016 18:48:00 +0200 (CEST) Subject: Re: Set non blocking on /dev/rdma_cm To: Alex Bowden , KrishnamRaju ErapaRaju , "freebsd-infiniband@freebsd.org" References: <516f3566-33f0-c1d6-ab2d-997290a16ae5@selasky.org> From: Hans Petter Selasky Message-ID: <20c22456-b057-2807-ff2e-124e7edde62a@selasky.org> Date: Wed, 17 Aug 2016 18:52:29 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Aug 2016 16:48:03 -0000 On 08/17/16 18:38, Alex Bowden wrote: > Hey Hans & Krishna, > > > I've applied the patches. I've got past the initial fcntl call. Do we need to add FIONWRITE and FIONREAD? > Only if your app is using them. They need to do more than return (0). --HPS From owner-freebsd-infiniband@freebsd.org Wed Aug 17 17:59:35 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E4FC9BBD8FB for ; Wed, 17 Aug 2016 17:59:35 +0000 (UTC) (envelope-from alex.bowden@outlook.com) Received: from BAY004-OMC2S22.hotmail.com (bay004-omc2s22.hotmail.com [65.54.190.97]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "*.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B49631964 for ; Wed, 17 Aug 2016 17:59:35 +0000 (UTC) (envelope-from alex.bowden@outlook.com) Received: from NAM01-SN1-obe.outbound.protection.outlook.com ([65.54.190.123]) by BAY004-OMC2S22.hotmail.com over TLS secured channel with Microsoft SMTPSVC(7.5.7601.23008); Wed, 17 Aug 2016 10:58:30 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=b8r0FWbPAS4s1LIhr3jxAVq8ExXXF9pdQCJrDfvYf08=; b=MQlZIBL/aYphh/vPKFWwFTn/LGFcHNJTQT6pTxq7blnm+CBtPWq3/Qv4ZLgcriPMjkI87FxCdXL9ryay81XKxyTLSUIQSXZQX3XZ656/A0Oog260CSl8AuQ+OvmgiBFwH22Q6AwIVCZNyUSINQGpnlfmea+90+io//FxnjeOuWVD5jQNC5quXmG1ZHpDGOK8gg/6XWUZFU75dNLBF+5CC2ipzXASawxRmb2dhFn1DmIUNDRK8DqRArKVi7BEU1hIIBZoPMLV8ZF6LTgO5cWf80yzRTD1Zzd6xTH4nwayFEG9SnymptLQla037qnAnmrhrRh6Vn/rVk4Oa+8eS9A9oQ== Received: from BN3NAM01FT057.eop-nam01.prod.protection.outlook.com (10.152.66.52) by BN3NAM01HT193.eop-nam01.prod.protection.outlook.com (10.152.67.230) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384) id 15.1.567.7; Wed, 17 Aug 2016 17:58:29 +0000 Received: from DM2PR05MB751.namprd05.prod.outlook.com (10.152.66.52) by BN3NAM01FT057.mail.protection.outlook.com (10.152.66.158) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384) id 15.1.577.8 via Frontend Transport; Wed, 17 Aug 2016 17:58:29 +0000 Received: from DM2PR05MB751.namprd05.prod.outlook.com ([10.141.178.148]) by DM2PR05MB751.namprd05.prod.outlook.com ([10.141.178.148]) with mapi id 15.01.0587.005; Wed, 17 Aug 2016 17:58:28 +0000 From: Alex Bowden To: "freebsd-infiniband@freebsd.org" Subject: rdma_bind_addr not allowing wildcarded address? Thread-Topic: rdma_bind_addr not allowing wildcarded address? Thread-Index: AQHR+LB3o+jmdv7k3k2zRKqJiaXwjw== Date: Wed, 17 Aug 2016 17:58:27 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=softfail (sender IP is 10.152.66.52) smtp.mailfrom=outlook.com; freebsd.org; dkim=none (message not signed) header.d=none;freebsd.org; dmarc=fail action=none header.from=outlook.com; received-spf: SoftFail (protection.outlook.com: domain of transitioning outlook.com discourages use of 10.152.66.52 as permitted sender) x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [oei3q7WYHvhaQtp6fjdzRUEoF/sptRBU] x-eopattributedmessage: 0 x-forefront-antispam-report: CIP:10.152.66.52; IPV:NLI; CTRY:; EFV:NLI; SFV:NSPM; SFS:(10019020)(98900003); DIR:OUT; SFP:1102; SCL:1; SRVR:BN3NAM01HT193; H:DM2PR05MB751.namprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; x-microsoft-exchange-diagnostics: 1; BN3NAM01HT193; 6:+IhqUxBtlbh6zFDsUfbap05IcgKSBzs5mF6FQht6/gYD0WpL/ZViQIbckroT1vBPVWy3gyFnLLBhGStZ9AR+GuHeJ4jhcMMmdfrgs/HdJSIcyYDpCV0jxvdLXdlU6Fj4d5zR2rsOPUfZfWCcS4WjtucB0W2tVOrVBh3M04Mxmyw6nV4vK4VuHMlrc67f7JJJXx6urcrLKjvvdr9E9SsHByM1netykF1TA4/ozV1KRz7Wd62Z+TD2k+XIwJnrdCDSr6p1Wu7d1oW7+zF8py9KG/NnhXSeyiyuQeT8r5+puO16MrJsRBRpMxjItPU2zk9U; 5:JzeewcYadS+IVDmyDXkKKpu/k/ry6osFLKlsB3nSost1sPAcd3eBiA6CSrXfBTv3MvLIH5ybdCL8jfoYZQVkCeO8hjAVXtwKxFxiDKv76x0U+5UEmC5ovYhS9BAckQEA+3nQBI8Yz9qF3PE2fKJtHA==; 24:txWHrXgO1HIP688+zWB1V0++G0l6Ub4qsrsy82ev/PSueWgQVmjVEQhquuH0UGZPd0eGgylQX65BVJ+V8dTEzdAaKHx/pcrSI42SzrQ+jZ8=; 7:LhFhqUnRDnXvIRzCorjuuzRyjKB405h8utP6qu2rXJhSnC0UWPSM4sOSGrA/dRmZ5J7CfZDgWBlQJh0bMlKmTobFcrmXBc+lu7lPRY2xwFjaPrzBXQQp4eSL8CygTxhKAR4M4XnPk0rlY5AUKNnuncs/VpdW1Cc07OJwvaAxWt0Z2znPIPy1IDYDpRJOvYX8cWP8ciVpG2WiI7L/B08e1Us74Z8qub1VNiXTGjZdAR/a8XUo352jrIgsLgT8HzKP x-ms-office365-filtering-correlation-id: 3218bb73-d156-4244-1707-08d3c6c817a1 x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(1601124038)(1601125047); SRVR:BN3NAM01HT193; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(432015012)(82015046); SRVR:BN3NAM01HT193; BCL:0; PCL:0; RULEID:; SRVR:BN3NAM01HT193; x-forefront-prvs: 0037FD6480 spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-originalarrivaltime: 17 Aug 2016 17:58:27.9392 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Internet X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN3NAM01HT193 X-OriginalArrivalTime: 17 Aug 2016 17:58:30.0602 (UTC) FILETIME=[F69F42A0:01D1F8B0] Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Aug 2016 17:59:36 -0000 Hey Hans, Here's another one. I believe with rdma_bind_addr you do not need to specif= y an ip?: ./rping -s -v -d -C 10 -a 192.168.188.6 -p 4420 -P count 10 port 4420 libibverbs: Warning: couldn't open config directory '/etc/ibverbs/'. created cm_id 0x801c37000 rdma_bind_addr successful rdma_listen ./rping -s -v -d -C 10 -p 4420 -P count 10 port 4420 libibverbs: Warning: couldn't open config directory '/etc/ibverbs/'. created cm_id 0x801c37000 rdma_bind_addr: Protocol not supported destroy cm_id 0x801c37000 I have another app that also tries the same thing and gets back "Protocol n= ot supported." I am using RoCE V1 at the moment on MLX4. Cheers, Alex From owner-freebsd-infiniband@freebsd.org Wed Aug 17 19:26:34 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3DB05BBD5D9 for ; Wed, 17 Aug 2016 19:26:34 +0000 (UTC) (envelope-from Krishna2@chelsio.com) Received: from NAM02-BL2-obe.outbound.protection.outlook.com (mail-bl2nam02on0120.outbound.protection.outlook.com [104.47.38.120]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DE8771A9A for ; Wed, 17 Aug 2016 19:26:32 +0000 (UTC) (envelope-from Krishna2@chelsio.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chelsious.onmicrosoft.com; s=selector1-chelsio-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=v31/x5Xs4xpPzJYjCVXS34loWA0PiE/Nm8aiJCG6qwI=; b=pecsOdEptfJ7UCTEnJiJYAdsJZ96D9idHymn53ukFynuBYm5MeHz18tJ6tO70IPPEwq+lHaHkUf0czHmt/DNIqZGFNNojun7YbqgNftCYIntPEefsgPS+nt1UAkLkX5udqtbPxIrEEi5xzmxpabS1j+dGVVWy/Isc6fKnjxChdA= Received: from BN4PR12MB0818.namprd12.prod.outlook.com (10.164.58.152) by BN4PR12MB0818.namprd12.prod.outlook.com (10.164.58.152) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384) id 15.1.587.9; Wed, 17 Aug 2016 13:52:02 +0000 Received: from BN4PR12MB0818.namprd12.prod.outlook.com ([10.164.58.152]) by BN4PR12MB0818.namprd12.prod.outlook.com ([10.164.58.152]) with mapi id 15.01.0587.009; Wed, 17 Aug 2016 13:52:02 +0000 From: KrishnamRaju ErapaRaju To: Hans Petter Selasky , Alex Bowden , "freebsd-infiniband@freebsd.org" Subject: RE: Set non blocking on /dev/rdma_cm Thread-Topic: Set non blocking on /dev/rdma_cm Thread-Index: AQHR9/Ra19SP/34IP0+fVdY/IpOON6BMt7qwgAAspGeAAEcaYA== Date: Wed, 17 Aug 2016 13:52:01 +0000 Message-ID: References: <516f3566-33f0-c1d6-ab2d-997290a16ae5@selasky.org> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Krishna2@chelsio.com; x-originating-ip: [111.93.130.157] x-ms-office365-filtering-correlation-id: 18a57e1e-a4eb-4491-9b27-08d3c6a5aa93 x-microsoft-exchange-diagnostics: 1; BN4PR12MB0818; 6:AxzyMIEHM7mcNDCSJrwKF5uAsCMXW3+2MRHsmA5q9znHEPMNPvUeMtQmwNGL1Tn2aqrvNAtrw7sODlW1kT4bFAOxwC6YHLyLX87iqLj6I4iu2XSzxwzeXrOos8YlFt55uOsx5M0uAl1RO4Hr0a7HBOZxgmj0nDlSxbJWnWKMSG6+b5UNSdz0E8o4VscL/DDwkdOTUOc3bpUxyXLVNp2NYvnuI953uhFtHjo17T5+H0FNenEX4jIHkgtd5froUogq4aAQQyWg3Vb3FVRZq33hjIOtvY9CQQ+f8e30ebCcMgLePWP1X1NoTbKT0DOOcVjd; 5:ANVOWFLR9H/SWzf6at62uv5ehFqhO0QMI9LiPJSpZAnMaimWO3iQUsXJK4UWvBzmDPOtgxuETobuiyYyIDP9pVlmNvRec6CjMQ/F9hRnt2LOsqGpXbuKzt0f5mw6EutgfvVdfMlrobPECwmL1CX77A==; 24:FGVAgoQUfK7I1DfvkR9rcr6Xp/HP2yzfkEuymjEsMwpV3h8UV0oRpUdVPSgswWpqJUUnMjLgRX3DNir2Ym6XoInXJquis69NcIKpN4PKrOo=; 7:bhO7p4Weo2ubJ2bOGtAaSyZWOpgphuMjnDo5XA8w8PXzJR8/TkUw68yVkB8lXoyZJQvUJ6IBGKjz0ItgiCYRGjybGlrtvUaI+D/m85TNRaBQk8maUqmNnNk2hdDd8TNvehy8ETn0C3AKnf5aRrOQwRtdoXsoRerKVxavN7YvLRtAEU3gvuDqSuzR1vk+S/TBxqRhk88nIbGbigWZQnhvHkuP1H2h4shuXPYiwf4nPlJM18eI4W1Xy7nuKg7Zq1lA x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BN4PR12MB0818; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(189930954265078); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040176)(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001)(6042046)(6043046); SRVR:BN4PR12MB0818; BCL:0; PCL:0; RULEID:; SRVR:BN4PR12MB0818; x-forefront-prvs: 0037FD6480 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(979002)(6009001)(7916002)(377454003)(199003)(24454002)(189002)(13464003)(10400500002)(66066001)(19580395003)(19580405001)(305945005)(54356999)(106356001)(74316002)(7846002)(7696003)(7736002)(5002640100001)(5001770100001)(86362001)(97736004)(76176999)(81166006)(107886002)(50986999)(189998001)(101416001)(68736007)(93886004)(8676002)(3660700001)(11100500001)(2900100001)(33656002)(87936001)(3846002)(2950100001)(6116002)(122556002)(77096005)(102836003)(586003)(92566002)(2906002)(3280700002)(81156014)(99286002)(105586002)(106116001)(8936002)(9686002)(5890100001)(2501003)(76576001)(969003)(989001)(999001)(1009001)(1019001); DIR:OUT; SFP:1102; SCL:1; SRVR:BN4PR12MB0818; H:BN4PR12MB0818.namprd12.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; received-spf: None (protection.outlook.com: chelsio.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: chelsio.com X-MS-Exchange-CrossTenant-originalarrivaltime: 17 Aug 2016 13:52:02.0463 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 065db76d-a7ae-4c60-b78a-501e8fc17095 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN4PR12MB0818 X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Aug 2016 19:26:34 -0000 Hi Hans, The error "Inappropriate ioctl for device" is not occurring after applying = your patch. Will check whether rdma_get_cm_event() is blocking or not when fcntl(O_NONB= LOCK) is set. Thanks, Krishna. -----Original Message----- From: Hans Petter Selasky [mailto:hps@selasky.org]=20 Sent: Wednesday, August 17, 2016 3:06 PM To: KrishnamRaju ErapaRaju ; Alex Bowden ; freebsd-infiniband@freebsd.org Subject: Re: Set non blocking on /dev/rdma_cm On 08/17/16 11:30, Hans Petter Selasky wrote: > On 08/17/16 09:03, KrishnamRaju ErapaRaju wrote: >> Hi Alex, >> >> >> I too faced the same issue earlier. But could not look into=20 >> it/report it due to other priorities. >> >> Hans, if you have root caused it please point me to the patch, few of=20 >> my apps uses O_NONBLOCK. >> > > Hi Alex, > > Can you try the attached patch? > > cd /usr/src > cat ibcore_fionbio.diff | patch -p1 > > --HPS > Hi, Looks like we also need to support FIOASYNC. Updated patch attached. --HPS From owner-freebsd-infiniband@freebsd.org Thu Aug 18 08:29:46 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E63F4BB620B for ; Thu, 18 Aug 2016 08:29:46 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (mail.turbocat.net [IPv6:2a01:4f8:d16:4514::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B293B19F1 for ; Thu, 18 Aug 2016 08:29:46 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id EDC131FE022; Thu, 18 Aug 2016 10:29:44 +0200 (CEST) Subject: Re: rdma_bind_addr not allowing wildcarded address? To: Alex Bowden , "freebsd-infiniband@freebsd.org" References: From: Hans Petter Selasky Message-ID: <363fe77e-a166-27f5-f08c-481b22b3dc58@selasky.org> Date: Thu, 18 Aug 2016 10:34:14 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Aug 2016 08:29:47 -0000 On 08/17/16 19:58, Alex Bowden wrote: > Hey Hans, > > Here's another one. I believe with rdma_bind_addr you do not need to specify an ip?: > > ./rping -s -v -d -C 10 -a 192.168.188.6 -p 4420 -P > count 10 > port 4420 > libibverbs: Warning: couldn't open config directory '/etc/ibverbs/'. > created cm_id 0x801c37000 > rdma_bind_addr successful > rdma_listen > > ./rping -s -v -d -C 10 -p 4420 -P > count 10 > port 4420 > libibverbs: Warning: couldn't open config directory '/etc/ibverbs/'. > created cm_id 0x801c37000 > rdma_bind_addr: Protocol not supported > destroy cm_id 0x801c37000 > > I have another app that also tries the same thing and gets back "Protocol not supported." I am using RoCE V1 at the moment on MLX4. > Hi, Do you have a sysctl called "unify_tcp_port_space" ? sysctl -a | grep unify_tcp_port_space Try to toggle its value from 1 to 0. Does it make any difference? --HPS From owner-freebsd-infiniband@freebsd.org Thu Aug 18 08:49:57 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F3F3CBB6725 for ; Thu, 18 Aug 2016 08:49:56 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (mail.turbocat.net [IPv6:2a01:4f8:d16:4514::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BC7BE16B5 for ; Thu, 18 Aug 2016 08:49:56 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 150241FE022; Thu, 18 Aug 2016 10:49:55 +0200 (CEST) Subject: Re: Set non blocking on /dev/rdma_cm To: Alex Bowden , KrishnamRaju ErapaRaju , "freebsd-infiniband@freebsd.org" References: <516f3566-33f0-c1d6-ab2d-997290a16ae5@selasky.org> <20c22456-b057-2807-ff2e-124e7edde62a@selasky.org> From: Hans Petter Selasky Message-ID: <93379a43-1839-80cf-02f9-6e40b6741090@selasky.org> Date: Thu, 18 Aug 2016 10:54:24 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.0 MIME-Version: 1.0 In-Reply-To: <20c22456-b057-2807-ff2e-124e7edde62a@selasky.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Aug 2016 08:49:57 -0000 FYI: https://svnweb.freebsd.org/changeset/base/304342 --HPS From owner-freebsd-infiniband@freebsd.org Thu Aug 18 14:52:12 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 45321BBD67A for ; Thu, 18 Aug 2016 14:52:12 +0000 (UTC) (envelope-from alexander.e.bowden@gmail.com) Received: from mail-wm0-x22a.google.com (mail-wm0-x22a.google.com [IPv6:2a00:1450:400c:c09::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CE2901F79 for ; Thu, 18 Aug 2016 14:52:11 +0000 (UTC) (envelope-from alexander.e.bowden@gmail.com) Received: by mail-wm0-x22a.google.com with SMTP id o80so35877646wme.1 for ; Thu, 18 Aug 2016 07:52:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=DoOkKVvdWTnWnFnMhDFl7NI10GGWLfE/rBiCYkaLloM=; b=rfG5m9L33w54TD5GUVaE2wgEKvRmpi+d71e/WQECp636kgA6Skij6G5Kw1KDgCthqQ b0YhiepXti9Jry0mlC1FVcF6ZvivKnnHKA8ZNc6Whjv80FOZBc/UkjFMQPOYy+gUwS0r g4099W96uu8/cPuXq5IZMe5REN5ommz08znbnZLPyXbWkJqiGWwavq46LNk8/p01jQTj p8ltODq4THcJoSHyC2/CEiXgJCa4XVdgiF3ChPFa95N8YqF+nXVaZ2nkpwO9YLP3uPE6 jtzioj1Sh3YkV0DA5yVAdOl+NqdfYNTAsXBNAcIiORBqXjK8OC9gWDE0on56qV9GWgUJ jPFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=DoOkKVvdWTnWnFnMhDFl7NI10GGWLfE/rBiCYkaLloM=; b=hs1dOf/bR2qOpm9MiOgPF8FGbeEO6lxBFK1Xlv1wOBsxivrn1OepoKRt68CbOyC2in 7y5rH9o89bKZSmQfEDD3/O/uMo/YPmMAw9Cftax0ToAElYZwm+sNCRE7FAjxpYxEM/Ku mrny2q+Oyu+lnuENGONpomBleDVAC4HgWn6YgqXw7U9SrGAvXQMzbmRMn48an3FNSocu B2c/Jlz50YGfFeStMjJbkaq6Dw1SEuDz/8cgtafKr3RXmFOnYDayV/+YFFP32YpB2WLy BiuvJi6ykgEg82l3HsTd66IH6hSNhBqffafKlHUTYyH4iG9NmXNy4UlpNKESK38SU7xr jCBw== X-Gm-Message-State: AEkoouvoRkJF7RAeahwPcUNU3g7vAfPMLL+tB3TgCoSeCDzKwtaIKV011mlijbRFHxoaLsX46P3KJlHHpVJ7aw== X-Received: by 10.195.13.18 with SMTP id eu18mr2466805wjd.121.1471531930179; Thu, 18 Aug 2016 07:52:10 -0700 (PDT) MIME-Version: 1.0 Received: by 10.194.5.134 with HTTP; Thu, 18 Aug 2016 07:52:09 -0700 (PDT) In-Reply-To: <363fe77e-a166-27f5-f08c-481b22b3dc58@selasky.org> References: <363fe77e-a166-27f5-f08c-481b22b3dc58@selasky.org> From: Alex Bowden Date: Thu, 18 Aug 2016 10:52:09 -0400 Message-ID: Subject: Re: rdma_bind_addr not allowing wildcarded address? To: Hans Petter Selasky Cc: "freebsd-infiniband@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Aug 2016 14:52:12 -0000 Success! That did the trick. Cheers, Alex On Thu, Aug 18, 2016 at 4:34 AM, Hans Petter Selasky wrote: > On 08/17/16 19:58, Alex Bowden wrote: > > Hey Hans, > > > > Here's another one. I believe with rdma_bind_addr you do not need to > specify an ip?: > > > > ./rping -s -v -d -C 10 -a 192.168.188.6 -p 4420 -P > > count 10 > > port 4420 > > libibverbs: Warning: couldn't open config directory '/etc/ibverbs/'. > > created cm_id 0x801c37000 > > rdma_bind_addr successful > > rdma_listen > > > > ./rping -s -v -d -C 10 -p 4420 -P > > count 10 > > port 4420 > > libibverbs: Warning: couldn't open config directory '/etc/ibverbs/'. > > created cm_id 0x801c37000 > > rdma_bind_addr: Protocol not supported > > destroy cm_id 0x801c37000 > > > > I have another app that also tries the same thing and gets back > "Protocol not supported." I am using RoCE V1 at the moment on MLX4. > > > > Hi, > > Do you have a sysctl called "unify_tcp_port_space" ? > > sysctl -a | grep unify_tcp_port_space > > Try to toggle its value from 1 to 0. Does it make any difference? > > --HPS > >