Date: Wed, 15 Aug 2018 13:19:12 +0100 From: "Kristof Provost" <kp@FreeBSD.org> To: "Matthew Macy" <mmacy@freebsd.org> Cc: freebsd-net@freebsd.org Subject: Re: Panic during ci test run Message-ID: <EC73F468-5B93-41CE-910E-CC171125E2C4@FreeBSD.org> In-Reply-To: <CAPrugNpDu-y1y1jOv6t3ibb=zLO91Sf7k0XXH1YYVZz8YTvP6Q@mail.gmail.com> References: <34C6043C-FDD2-4812-AFF3-C61DEF7AE435@FreeBSD.org> <CAPrugNpDu-y1y1jOv6t3ibb=zLO91Sf7k0XXH1YYVZz8YTvP6Q@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
With your mmacy/projects/mcastfix branch I can no longer reproduce the
panic.
Regards,
Kristof
On 14 Aug 2018, at 23:42, Matthew Macy wrote:
> This isn't reproducing it for me. I'll need more specifics on your
> configuration.
> -M
>
> On Sat, Aug 11, 2018 at 2:04 AM Kristof Provost <kp@freebsd.org>
> wrote:
>
>> The fibs_test:subnet_route_with_multiple_fibs_on_same_subnet test
>> (/usr/tests/sys/netinet/) consistently provokes a panic.
>>
>> Note that this requires:
>>
>> - test_suites.FreeBSD.fibs = '1 2' in
>> /usr/local/etc/kyua/kyua.conf
>> - net.fibs=3 in /boot/loader.conf
>> - sysctl net.add_addr_allfibs=0
>>
>> Then:
>>
>> - cd /usr/tests/sys/netinet/
>> - sudo kyua test
>>
>> This results in:
>>
>> Fatal trap 9: general protection fault while in kernel mode
>> cpuid = 2; apic id = 02
>> instruction pointer = 0x20:0xffffffff80ded4c3
>> stack pointer = 0x28:0xfffffe0000427860
>> frame pointer = 0x28:0xfffffe00004278a0
>> code segment = base 0x0, limit 0xfffff, type 0x1b
>> = DPL 0, pres 1, long 1, def32 0, gran 1
>> processor eflags = interrupt enabled, resume, IOPL = 0
>> current process = 0 (softirq_2)
>> [ thread pid 0 tid 100021 ]
>> Stopped at inp_gcmoptions+0xe3: movq ll+0x33f(%rax),%r9
>> db> bt
>> Tracing pid 0 tid 100021 td 0xfffff80004605000
>> inp_gcmoptions() at inp_gcmoptions+0xe3/frame 0xfffffe00004278a0
>> epoch_call_task() at epoch_call_task+0x21a/frame 0xfffffe00004278f0
>> gtaskqueue_run_locked() at gtaskqueue_run_locked+0x139/frame
>> 0xfffffe0000427940
>> gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x88/frame
>> 0xfffffe0000427970
>> fork_exit() at fork_exit+0x84/frame 0xfffffe00004279b0
>> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00004279b0
>> --- trap 0, rip = 0, rsp = 0, rbp = 0 ---
>>
>> kgdb decodes that to:
>>
>> #0 __curthread () at ./machine/pcpu.h:230
>> #1 doadump (textdump=0) at /usr/src/sys/kern/kern_shutdown.c:366
>> #2 0xffffffff8043dd4b in db_dump (dummy=<optimized out>,
>> dummy2=<unavailable>, dummy3=<unavailable>, dummy4=<unavailable>) at
>> /usr/src/sys/ddb/db_command.c:574
>> #3 0xffffffff8043db19 in db_command (last_cmdp=<optimized out>,
>> cmd_table=<optimized out>, dopager=<optimized out>) at
>> /usr/src/sys/ddb/db_command.c:481
>> #4 0xffffffff8043d894 in db_command_loop () at
>> /usr/src/sys/ddb/db_command.c:534
>> #5 0xffffffff80440abf in db_trap (type=<optimized out>,
>> code=<optimized out>) at /usr/src/sys/ddb/db_main.c:252
>> #6 0xffffffff80bdef43 in kdb_trap (type=9, code=0, tf=<optimized
>> out>) at /usr/src/sys/kern/subr_kdb.c:693
>> #7 0xffffffff8107aee1 in trap_fatal (frame=0xfffffe00004277a0,
>> eva=0) at /usr/src/sys/amd64/amd64/trap.c:906
>> #8 0xffffffff8107a3bd in trap (frame=0xfffffe00004277a0) at
>> /usr/src/sys/amd64/amd64/trap.c:203
>> #9 <signal handler called>
>> #10 inp_gcmoptions (ctx=0xfffff800142da5e0) at
>> /usr/src/sys/netinet6/in6_mcast.c:1650
>> #11 0xffffffff80bd9c7a in epoch_call_task (arg=<optimized out>) at
>> /usr/src/sys/kern/subr_epoch.c:507
>> #12 0xffffffff80bdd069 in gtaskqueue_run_locked
>> (queue=0xfffff800040ceb00) at /usr/src/sys/kern/subr_gtaskqueue.c:332
>> #13 0xffffffff80bdcde8 in gtaskqueue_thread_loop (arg=<optimized
>> out>) at /usr/src/sys/kern/subr_gtaskqueue.c:507
>> #14 0xffffffff80b53084 in fork_exit (callout=0xffffffff80bdcd60
>> <gtaskqueue_thread_loop>, arg=0xfffffe0087e40038,
>> frame=0xfffffe00004279c0) at /usr/src/sys/kern/kern_fork.c:1057
>> #15 <signal handler called>
>>
>> It looks like the inm has been freed at that point, so we try to
>> dereference a freed pointer, and that doesn’t go well for us:
>>
>> (kgdb) fr 10
>> #10 inp_gcmoptions (ctx=0xfffff800142da5e0) at
>> /usr/src/sys/netinet6/in6_mcast.c:1650
>> 1650 CURVNET_SET(ifp->if_vnet);
>> (kgdb) p ifp
>> $1 = (struct ifnet *) 0xdeadc0dedeadc0de
>> (kgdb)
>> (kgdb) l
>> 1645 if (imf)
>> 1646 im6f_leave(imf);
>> 1647 inm = imo->im6o_membership[idx];
>> 1648 ifp = inm->in6m_ifp;
>> 1649 if (ifp != NULL) {
>> 1650 CURVNET_SET(ifp->if_vnet);
>> 1651 (void)in6_leavegroup(inm, imf);
>> 1652 CURVNET_RESTORE();
>> 1653 } else {
>> 1654 (void)in6_leavegroup(inm, imf);
>> (kgdb) p inm
>> $2 = (struct in6_multi *) 0xfffff8001435b200
>> (kgdb) p *inm
>> $3 = {in6m_addr = {__u6_addr = {__u6_addr8 =
>> "\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255",
>> <incomplete sequence \336>, __u6_addr16 = {49374, 57005, 49374,
>> 57005, 49374, 57005, 49374, 57005}, __u6_addr32 =
>> {3735929054, 3735929054, 3735929054, 3735929054}}}, in6m_ifp =
>> 0xdeadc0dedeadc0de, in6m_ifma = 0xdeadc0dedeadc0de,
>> in6m_refcount = 3735929054, in6m_state = 3735929054, in6m_timer =
>> 3735929054, in6m_mli = 0xdeadc0dedeadc0de, in6m_nrele = {sle_next =
>> 0xdeadc0dedeadc0de}, in6m_srcs = {
>> rbh_root = 0xdeadc0dedeadc0de}, in6m_nsrc = 16045693110842147038,
>> in6m_scq = {mq_head = {stqh_first = 0xdeadc0dedeadc0de, stqh_last =
>> 0xdeadc0dedeadc0de},
>> mq_len = -559038242, mq_maxlen = -559038242}, in6m_lastgsrtv =
>> {tv_sec = -2401050962867404578, tv_usec = -2401050962867404578},
>> in6m_sctimer = 49374, in6m_scrv = 57005,
>> in6m_st = {{iss_fmode = 49374, iss_asm = 57005, iss_ex = 49374,
>> iss_in = 57005, iss_rec = 49374}, {iss_fmode = 57005, iss_asm =
>> 49374, iss_ex = 57005, iss_in = 49374,
>> iss_rec = 57005}}}
>> (kgdb)
>> (kgdb) p nmships
>> $4 = 1
>> (kgdb) p *imf
>> $6 = {im6f_sources = {rbh_root = 0x0}, im6f_nsrc = 0, im6f_st =
>> "\002\001"}
>> (kgdb)
>>
>> Regards,
>> Kristof
>>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?EC73F468-5B93-41CE-910E-CC171125E2C4>