Date: Wed, 26 Feb 2020 22:11:57 +0100 From: Andreas Kempe <kempe@lysator.liu.se> Cc: freebsd-infiniband@freebsd.org Subject: Re: [PATCH]: ipoib with mlx4 initialisation ordering Message-ID: <20200226211157.GF22659@moira.hest-guild.se> In-Reply-To: <20200226210554.GE22659@moira.hest-guild.se> References: <20200222004838.GA22659@moira.hest-guild.se> <9d76992b-6ba4-2419-61ff-5035aa45e597@selasky.org> <20200224194608.GC22659@moira.hest-guild.se> <16883d49-3cc0-d9cc-0877-46f811eeb8f1@selasky.org> <20200226210554.GE22659@moira.hest-guild.se>
next in thread | previous in thread | raw e-mail | index | archive | help
--9/eUdp+dLtKXvemk Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Feb 26, 2020 at 10:05:55PM +0100, Andreas Kempe wrote: > On Mon, Feb 24, 2020 at 11:50:10PM +0100, Hans Petter Selasky wrote: > > Hi, > >=20 > > On 2020-02-24 20:46, Andreas Kempe wrote: > > > If you want me to try reordering the deinitialisation, I should be > > > able to do that this coming wednesday. > > >=20 > >=20 > > Yes, please. > >=20 > > Depending on how lists work, might not be so good in the long run. > >=20 >=20 > First I tried the following change and the machine still hung. >=20 > > --- sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_main.c (revision 356611) > > +++ sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_main.c (working copy) > > @@ -1739,7 +1739,7 @@ > > } > > =20 > > module_init(ipoib_init_module); > > -module_exit(ipoib_cleanup_module); > > +module_exit_order(ipoib_cleanup_module, SI_ORDER_FIRST); > > =20 > > static int > > ipoib_evhand(module_t mod, int event, void *arg) >=20 > Then I tried moving the mlx4 driver unloading using the following > change and the machine still hung. >=20 > > --- sys/dev/mlx4/mlx4_ib/mlx4_ib_main.c (revision 356611) > > +++ sys/dev/mlx4/mlx4_ib/mlx4_ib_main.c (working copy) > > @@ -3320,7 +3320,7 @@ > > } > > =20 > > module_init_order(mlx4_ib_init, SI_ORDER_THIRD); > > -module_exit(mlx4_ib_cleanup); > > +module_exit_order(mlx4_ib_cleanup, SI_ORDER_THIRD); > > =20 > > static int > > mlx4ib_evhand(module_t mod, int event, void *arg) >=20 > I don't really feel like analysing why it still hangs with the above > changes at the moment since we got something that works for us. If you > have any suggestions you want me to try, I could still do that. >=20 > > > > I'm currently trying to reproduce the issue. > >=20 > > > >=20 > > > We're seeing the issue every time when running the machine in a > > > network with a Linux machine. We simply need to send a bit of data on > > > the link and then trigger a shutdown. > >=20 > > I see. > >=20 >=20 > I can add that we compiled the modules into the kernel by adding the > following to the GENERIC kernel of 12.1-STABLE: >=20 > > # INFINIBAND > > options COMPAT_LINUXKPI > > options OFED > > options SDP > > options IPOIB_CM > >=20 > > device ipoib > > device mlx4 > > device mlx4ib >=20 > Adding freebsd-inifinband again. >=20 > Cordially, > Andreas Kempe --9/eUdp+dLtKXvemk Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEETci4cPcl+ZcyiACiCkqKrhcKSD0FAl5W3xcACgkQCkqKrhcK SD2X7g/+IK8KPUN/P/BmvJjr7qQXDEq5JkvwtVguXQuWEWAzbh6i3FSFXhwpZkJJ 3U4yoz5t9se7Le+GJ0n0nDGBWjFZOr8oBrDmzXarHck52+ClfTeOaucGdx5c78bG E+dMfw0wHaJX/FIfv2A6u1HecYICcIZCv1KHjsFIkqERcl9sHNZDisNZq6HoLUzA An1AqcYj9eiEg8cmx/d8gdA6UOHlW4B/yps/Nvk0Mh0RPSsUoz3DMnhZSw0rdHEZ qY3VgzRElGsWZ8NBQpHjTDm6+DQJNE1ZTMPyuOZpTbEr7JwVUcHPW91f17TczKmC 5a6rK/aGUxLUJlgNG+IJfSZf5dI1oTcDQk6xtztEntFj1Y++pTyJx0ou11gxTEX0 sdzWthDDQh8ccS8/SZBcAYryXoPhJoUg1kzmYuTxrHmRu0+GWhmsiIsOjI4/g58K xC9rbwH4bSUc/dzvnw5pXKoFr3F/BAfHlkJdcZsC6UM00OOy5V6La+zC7dxb0jBW IQQ5pD3g/dQQZKIzQyWRt2kbQTUwCLpIWQcLK3pH2TDJ8fY1oWgX7GBdkoCW7MHm 4/s+eISZcU13InvPj7hdSvNVVg3ceSLp4wAVCPW0hQoHrd1bsXwUDf6p0gBjcE22 iOwAa98tVeFw0vAHtde9xHBuu9puVst2G4qfOicIDqOnVv+7Hxg= =ip4Y -----END PGP SIGNATURE----- --9/eUdp+dLtKXvemk--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20200226211157.GF22659>