Date: Sat, 7 Jul 2018 00:49:09 +0000 From: "Somayajulu, David" <David.Somayajulu@cavium.com> To: "freebsd-drivers@freebsd.org" <freebsd-drivers@freebsd.org>, "freebsd-net@freebsd.org" <freebsd-net@freebsd.org> Subject: ib_unregister_device - OFED related question Message-ID: <MWHPR0701MB372139C2DA2DA94DD7F8FF95EE460@MWHPR0701MB3721.namprd07.prod.outlook.com>
next in thread | raw e-mail | index | archive | help
Hi All, ib_unregister_device<http://fxr.watson.org/fxr/ident?v=3DFREEBSD10;i=3Dib_u= nregister_device>() is typically called as part of the rdma driver module u= nload for a given device. As part of execution of this function it loops th= rough the following code: ofed/drivers/inifiniband/core/device.c void ib_unregister_device<http://fxr.watson.org/fxr/ident?v=3DFREEBSD10;i= =3Dib_unregister_device>(struct ib_device<http://fxr.watson.org/fxr/ident?v= =3DFREEBSD10;i=3Dib_device> *device<http://fxr.watson.org/fxr/ident?v=3DFRE= EBSD10;i=3Ddevice>) { : list_for_each_entry_reverse<http://fxr.watson.org/fxr/i= dent?v=3DFREEBSD10;i=3Dlist_for_each_entry_reverse>(client<http://fxr.watso= n.org/fxr/ident?v=3DFREEBSD10;i=3Dclient>, &client_list, list<http://fxr.wa= tson.org/fxr/ident?v=3DFREEBSD10;i=3Dlist>) if (client<http://fxr.watson.org/f= xr/ident?v=3DFREEBSD10;i=3Dclient>->remove) client<http://= fxr.watson.org/fxr/ident?v=3DFREEBSD10;i=3Dclient>->remove(device<http://fx= r.watson.org/fxr/ident?v=3DFREEBSD10;i=3Ddevice>); : } For IB clients, client->remove(device) calls cma_remove_one() (ofed/drivers= /inifiniband/core/cma.c) which ends up calling cma_process_remove(). cma_process_remove() calls wait_for_completion(&cma_dev->comp). Now wait_for_completion() can potentially cause the thread to sleep(). >From kern/kern_module.c: int module_unload<http://fxr.watson.org/fxr/ident?v=3DFREEBSD10;i=3Dmodule_unlo= ad>(module_t<http://fxr.watson.org/fxr/ident?v=3DFREEBSD10;i=3Dmodule_t>; mo= d<http://fxr.watson.org/fxr/ident?v=3DFREEBSD10;i=3Dmod>) { int error<http://fxr.watson.org/fxr/ident?v=3DFREEBSD10;i=3Derror>= ; mtx_lock<http://fxr.watson.org/fxr/ident?v=3DFREEBSD10;i=3Dmtx_loc= k>(&Giant<http://fxr.watson.org/fxr/ident?v=3DFREEBSD10;i=3DGiant>); error<http://fxr.watson.org/fxr/ident?v=3DFREEBSD10;i=3Derror>; =3D= MOD_EVENT<http://fxr.watson.org/fxr/ident?v=3DFREEBSD10;i=3DMOD_EVENT>(mod= <http://fxr.watson.org/fxr/ident?v=3DFREEBSD10;i=3Dmod>, MOD_UNLOAD<http://= fxr.watson.org/fxr/ident?v=3DFREEBSD10;i=3DMOD_UNLOAD>); mtx_unlock<http://fxr.watson.org/fxr/ident?v=3DFREEBSD10;i=3Dmtx_u= nlock>(&Giant<http://fxr.watson.org/fxr/ident?v=3DFREEBSD10;i=3DGiant>); return (error<http://fxr.watson.org/fxr/ident?v=3DFREEBSD10;i=3Der= ror>); } We see that module_unload() grabs the Gaint Lock prior to invoking UNLOAD. = Isn't this a problem with cma_process_remove() or am I missing something? Thanks David S. (davidcs@freebsd.org<mailto:davidcs@freebsd.org>)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?MWHPR0701MB372139C2DA2DA94DD7F8FF95EE460>