Skip site navigation (1)Skip section navigation (2) 
Date:      Sat, 7 Jul 2018 00:49:09 +0000
From:      "Somayajulu, David" <David.Somayajulu@cavium.com>
To:        "freebsd-drivers@freebsd.org" <freebsd-drivers@freebsd.org>, "freebsd-net@freebsd.org" <freebsd-net@freebsd.org>
Subject:   ib_unregister_device - OFED related question
Message-ID:  <MWHPR0701MB372139C2DA2DA94DD7F8FF95EE460@MWHPR0701MB3721.namprd07.prod.outlook.com>
index | next in thread | raw e-mail 

Hi All,

ib_unregister_device<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=ib_unregister_device>() is typically called as part of the rdma driver module unload for a given device. As part of execution of this function it loops through the following code:



ofed/drivers/inifiniband/core/device.c



void ib_unregister_device<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=ib_unregister_device>(struct ib_device<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=ib_device>; *device<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=device>)

{

                    :

                    list_for_each_entry_reverse<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=list_for_each_entry_reverse>(client<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=client>, &client_list, list<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=list>)

                                         if (client<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=client>->remove)

                                                             client<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=client>->remove(device<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=device>);

                    :

}



For IB clients, client->remove(device) calls cma_remove_one() (ofed/drivers/inifiniband/core/cma.c) which ends up calling cma_process_remove().



cma_process_remove() calls wait_for_completion(&cma_dev->comp).



Now wait_for_completion() can potentially cause the thread to sleep().



>From kern/kern_module.c:

int

module_unload<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=module_unload>(module_t<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=module_t>; mod<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=mod>)

{

         int error<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=error>;



         mtx_lock<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=mtx_lock>(&Giant<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=Giant>);

         error<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=error>; = MOD_EVENT<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=MOD_EVENT>(mod<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=mod>, MOD_UNLOAD<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=MOD_UNLOAD>);

         mtx_unlock<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=mtx_unlock>(&Giant<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=Giant>);

         return (error<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=error>);

}

We see that module_unload() grabs the Gaint Lock prior to invoking UNLOAD. Isn't this a problem with cma_process_remove() or am I missing something?



Thanks

David S. (davidcs@freebsd.org<mailto:davidcs@freebsd.org>)
help

Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?MWHPR0701MB372139C2DA2DA94DD7F8FF95EE460>