From owner-freebsd-net@freebsd.org Sat Jul 7 00:49:31 2018 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0C5221042E52; Sat, 7 Jul 2018 00:49:31 +0000 (UTC) (envelope-from David.Somayajulu@cavium.com) Received: from NAM05-BY2-obe.outbound.protection.outlook.com (mail-by2nam05on060d.outbound.protection.outlook.com [IPv6:2a01:111:f400:fe52::60d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT TLS CA 4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 34812783AA; Sat, 7 Jul 2018 00:49:30 +0000 (UTC) (envelope-from David.Somayajulu@cavium.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0yDav9o6G4dyz09dArYr5mVgJ5PARaBvSWtE2kPvMF8=; b=JUACAlMgiigAOTC0EVFr5X631/WfoUJTJ2yg76Lm4P/nQVZKYHg3qx3x2RevR/tEVVWbDufNZTsDeVP6t2eeJwuT6FIFLW2w9jDMVyI9PKWi6rOzySkiTX36TFRVQzoEjrf6SB5NwhtQupz8eZzngMN0N5JOacUNSTvBOS0P2/A= Received: from MWHPR0701MB3721.namprd07.prod.outlook.com (10.167.237.26) by MWHPR0701MB3817.namprd07.prod.outlook.com (10.167.237.149) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.930.19; Sat, 7 Jul 2018 00:49:09 +0000 Received: from MWHPR0701MB3721.namprd07.prod.outlook.com ([fe80::e1da:5498:fc78:938e]) by MWHPR0701MB3721.namprd07.prod.outlook.com ([fe80::e1da:5498:fc78:938e%3]) with mapi id 15.20.0930.016; Sat, 7 Jul 2018 00:49:09 +0000 From: "Somayajulu, David" To: "freebsd-drivers@freebsd.org" , "freebsd-net@freebsd.org" Subject: ib_unregister_device - OFED related question Thread-Topic: ib_unregister_device - OFED related question Thread-Index: AdQVinFf9cg/j8gKSvSAFTbo/BlNLg== Date: Sat, 7 Jul 2018 00:49:09 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=David.Somayajulu@cavium.com; x-originating-ip: [198.186.1.6] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; MWHPR0701MB3817; 7:W2rkhJzfLaREfVOU1RODq7PDTHhYQ9hhxojyvJp2NsVG/4HfEUAnQyuC7LyYnHQzrMKR62DWIZwIP3IFLrWJKj2AVyUe6EYYPrdWDv91+hdhh+fHio3R8qamRhru4XxMtQ6sKypUCpAoM1a7lq7A15imbovZ7QKdS2WmBdl9gOIOVHZ2KKiNZ3vQvwekA0VpxEZpY0cdWQ2ZmR+Dw0J+2eifDU+cVc3bXka6MuJxQqLotnJBDen4sTQK/kniYc5x x-ms-exchange-antispam-srfa-diagnostics: SOS; x-ms-office365-filtering-correlation-id: 93b39707-2061-4f6c-b896-08d5e3a3737a x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989117)(4534165)(4627221)(201703031133081)(201702281549075)(8990107)(5600053)(711020)(2017052603328)(7153060)(7193020); SRVR:MWHPR0701MB3817; x-ms-traffictypediagnostic: MWHPR0701MB3817: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(28532068793085)(21748063052155); x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(3002001)(93006095)(93001095)(3231291)(944501410)(52105095)(10201501046)(149027)(150027)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(20161123560045)(20161123564045)(20161123558120)(6072148)(201708071742011)(7699016); SRVR:MWHPR0701MB3817; BCL:0; PCL:0; RULEID:; SRVR:MWHPR0701MB3817; x-forefront-prvs: 0726B2D7A6 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(396003)(39850400004)(366004)(376002)(346002)(136003)(199004)(189003)(53754006)(14454004)(86362001)(6436002)(450100002)(54896002)(8676002)(53936002)(478600001)(7696005)(7736002)(316002)(8936002)(72206003)(25786009)(97736004)(6116002)(5250100002)(2501003)(99286004)(606006)(790700001)(2906002)(102836004)(26005)(9326002)(186003)(6506007)(110136005)(66066001)(2900100001)(3846002)(33656002)(81156014)(68736007)(81166006)(9686003)(6306002)(236005)(74316002)(55016002)(486006)(476003)(5660300001)(7066003)(256004)(105586002)(106356001)(14444005); DIR:OUT; SFP:1101; SCL:1; SRVR:MWHPR0701MB3817; H:MWHPR0701MB3721.namprd07.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: wkP1NWFC5PYW4/fwNOEz85eYbX+Fbx3syG7+TrpVVvHDv4eoa0Qgtoc2RxWE9AqWJG+FAG+9zQXwq51UuePPWv2JXbs+X6YNDLXncxVXXCAID5/1Tne2IXG7SXP4a0re6svduQJ+OPCIbiMyaLlVBHd3vLOXBsDwQGEtv+Wuz1we1NMo2I+4aVnjuuLpCLb0fjSgNQbvvI42z1Xk90nIlfsizMLv65AUo2lJi1hr8A4HCHqMLbcYRvJEBSY1o7TU1IFCELUxZBtXJnbA12AVzidPf3EcezPh1q1ZvCNZCuhpzq8EqNP9yVM0ZCaJERGzrymsVHxUeaOUV77cVz1RwKCQSeaR1wrfUjkmW3MxP5M= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: cavium.com X-MS-Exchange-CrossTenant-Network-Message-Id: 93b39707-2061-4f6c-b896-08d5e3a3737a X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Jul 2018 00:49:09.7358 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR0701MB3817 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Jul 2018 00:49:31 -0000 Hi All, ib_unregister_device() is typically called as part of the rdma driver module u= nload for a given device. As part of execution of this function it loops th= rough the following code: ofed/drivers/inifiniband/core/device.c void ib_unregister_device(struct ib_device *device) { : list_for_each_entry_reverse(client, &client_list, list) if (client->remove) client->remove(device); : } For IB clients, client->remove(device) calls cma_remove_one() (ofed/drivers= /inifiniband/core/cma.c) which ends up calling cma_process_remove(). cma_process_remove() calls wait_for_completion(&cma_dev->comp). Now wait_for_completion() can potentially cause the thread to sleep(). >From kern/kern_module.c: int module_unload(module_t mo= d) { int error= ; mtx_lock(&Giant); error =3D= MOD_EVENT(mod= , MOD_UNLOAD); mtx_unlock(&Giant); return (error); } We see that module_unload() grabs the Gaint Lock prior to invoking UNLOAD. = Isn't this a problem with cma_process_remove() or am I missing something? Thanks David S. (davidcs@freebsd.org)