Date: Wed, 22 Jul 2020 06:54:17 +0200 From: Peter Libassi <peter@libassi.se> To: John-Mark Gurney <jmg@funkthat.com> Cc: Marko Zec <zec@fer.hr>, freebsd-net@freebsd.org, freebsd-current@freebsd.org Subject: Re: somewhat reproducable vimage panic Message-ID: <38F5A3A6-B578-4BA4-8F69-C248163CB6E0@libassi.se> In-Reply-To: <20200721202323.GE4213@funkthat.com> References: <20200721091654.GC4213@funkthat.com> <20200721113153.42d83119@x23> <20200721202323.GE4213@funkthat.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Is this related to=20 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D234985 = <https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D234985>; and = https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D238326 = <https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D238326>; /Peter > 21 juli 2020 kl. 22:23 skrev John-Mark Gurney <jmg@funkthat.com>: >=20 > Marko Zec wrote this message on Tue, Jul 21, 2020 at 11:31 +0200: >> On Tue, 21 Jul 2020 02:16:55 -0700 >> John-Mark Gurney <jmg@funkthat.com> wrote: >>=20 >>> I'm running: >>> FreeBSD test 13.0-CURRENT FreeBSD 13.0-CURRENT #0 r362596: Thu Jun = 25 >>> 05:02:51 UTC 2020 >>> = root@releng1.nyi.freebsd.org:/usr/obj/usr/src/amd64.amd64/sys/GENERIC >>> amd64 >>>=20 >>> and I'm working on improve the if_ure driver. I've put together a >>> little script that I've attached that I'm using to test the driver.. >>> It puts a couple ue interfaces each into their own jail, configures >>> them, and tries to pass traffic. This assumes that the two = interfaces >>> are connected together. >>>=20 >>> Pretty regularly when destroying the jails, I get the following >>> panic: CURVNET_SET at /usr/src/sys/netinet/in_mcast.c:626 >>> inm_release() curvnet=3D0 vnet=3D0xfffff80154c82a80 >>=20 >> Perhaps the attached patch could help? (disclaimer: not even >> compile-tested) >=20 > The patch compiled, but it just moved the panic earlier than before. >=20 > #4 0xffffffff80bc2123 in panic (fmt=3D<unavailable>) > at ../../../kern/kern_shutdown.c:839 > #5 0xffffffff80d61726 in inm_release_task (arg=3D<optimized out>,=20 > pending=3D<optimized out>) at ../../../netinet/in_mcast.c:633 > #6 0xffffffff80c2166a in taskqueue_run_locked = (queue=3D0xfffff800033cfd00) > at ../../../kern/subr_taskqueue.c:476 > #7 0xffffffff80c226e4 in taskqueue_thread_loop (arg=3D<optimized = out>) > at ../../../kern/subr_taskqueue.c:793 >=20 > Now it panics at the location of the new CURVNET_SET and not the > old one.. >=20 > Ok, decided to dump the contents of the vnet, and it looks like > it's a use after free: > (kgdb) print/x *(struct vnet *)0xfffff8012a283140 > $2 =3D {vnet_le =3D {le_next =3D 0xdeadc0dedeadc0de, le_prev =3D = 0xdeadc0dedeadc0de}, vnet_magic_n =3D 0xdeadc0de,=20 > vnet_ifcnt =3D 0xdeadc0de, vnet_sockcnt =3D 0xdeadc0de, vnet_state =3D = 0xdeadc0de, vnet_data_mem =3D 0xdeadc0dedeadc0de,=20 > vnet_data_base =3D 0xdeadc0dedeadc0de, vnet_shutdown =3D 0xde} >=20 > The patch did seem to make it happen quicker, or maybe I was just more > lucky this morning... >=20 >>> (kgdb) #0 __curthread () at = /usr/src/sys/amd64/include/pcpu_aux.h:55 >>> #1 doadump (textdump=3D1) at /usr/src/sys/kern/kern_shutdown.c:394 >>> #2 0xffffffff80bc6250 in kern_reboot (howto=3D260) >>> at /usr/src/sys/kern/kern_shutdown.c:481 >>> #3 0xffffffff80bc66aa in vpanic (fmt=3D<optimized out>, = ap=3D<optimized >>> out>) at /usr/src/sys/kern/kern_shutdown.c:913 >>> #4 0xffffffff80bc6403 in panic (fmt=3D<unavailable>) >>> at /usr/src/sys/kern/kern_shutdown.c:839 >>> #5 0xffffffff80d6553b in inm_release (inm=3D0xfffff80029043700) >>> at /usr/src/sys/netinet/in_mcast.c:630 >>> #6 inm_release_task (arg=3D<optimized out>, pending=3D<optimized = out>) >>> at /usr/src/sys/netinet/in_mcast.c:312 >>> #7 0xffffffff80c2521a in taskqueue_run_locked >>> (queue=3D0xfffff80003116b00) at = /usr/src/sys/kern/subr_taskqueue.c:476 >>> #8 0xffffffff80c26294 in taskqueue_thread_loop (arg=3D<optimized = out>) >>> at /usr/src/sys/kern/subr_taskqueue.c:793 >>> #9 0xffffffff80b830f0 in fork_exit ( >>> callout=3D0xffffffff80c26200 <taskqueue_thread_loop>,=20 >>> arg=3D0xffffffff81cf4f70 <taskqueue_thread>, >>> frame=3D0xfffffe0049e99b80) at /usr/src/sys/kern/kern_fork.c:1052 >>> #10 <signal handler called> >>> (kgdb)=20 >>>=20 >>> I have the core files so I can get additional information. >>>=20 >>> Let me know if you need any additional information. >>>=20 >>=20 >=20 >> Index: sys/netinet/in_mcast.c >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> --- sys/netinet/in_mcast.c (revision 363386) >> +++ sys/netinet/in_mcast.c (working copy) >> @@ -309,8 +309,10 @@ >> IN_MULTI_LOCK(); >> SLIST_FOREACH_SAFE(inm, &inm_free_tmp, inm_nrele, tinm) { >> SLIST_REMOVE_HEAD(&inm_free_tmp, inm_nrele); >> + CURVNET_SET(inm->inm_ifp->if_vnet); >> MPASS(inm); >> inm_release(inm); >> + CURVNET_RESTORE(); >> } >> IN_MULTI_UNLOCK(); >> } >=20 >=20 > --=20 > John-Mark Gurney Voice: +1 415 225 5579 >=20 > "All that I will do, has been done, All that I have, has not." > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?38F5A3A6-B578-4BA4-8F69-C248163CB6E0>