Date: Sat, 31 Oct 2015 09:46:03 -0400 From: Adrian Chadd <adrian@freebsd.org> To: "Alexander V. Chernikov" <melifaro@freebsd.org> Cc: freebsd-current <freebsd-current@freebsd.org>, FreeBSD Net <freebsd-net@freebsd.org> Subject: Re: panic in arptimer in r289937 Message-ID: <CAJ-Vmo=JjHonDqOYK%2BJDaf9581dRU5_KoaSTnY27JnzQm0v56w@mail.gmail.com> In-Reply-To: <2739461446298483@web2h.yandex.ru> References: <CAJ-VmonvVyTNuYv_as41yPCFdfR5T3FE45DP9MKAc-eyzXzPUg@mail.gmail.com> <2739461446298483@web2h.yandex.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
On 31 October 2015 at 09:34, Alexander V. Chernikov
<melifaro@freebsd.org> wrote:
>
>
> 31.10.2015, 05:32, "Adrian Chadd" <adrian@freebsd.org>:
>> Hiya,
>>
>> Here's a panic from arptimer:
> Hi Adrian,
>
> As far as I see, line 205 in if_ether.c is IF_AFDATA_LOCK(ifp) which happ=
ens after LLE_WUNLOCK().
> So, it looks like (pre-cached) ifp had been freed before locking ifdata.
> Do you have any more details on that? (e.g. was some interface detached a=
t that moment, is it reproducible, etc..)
>
> From a quick glance, potential use-after-free has been possible for quite=
a long time, but I wonder why it hasn't been observed before.
> Probably lltable_free() changes might have triggered that.
>
> I'll take a deeper look on that and reply.
Hiya!
Thanks for your quick response.
I mean, I use wifi, and ARPs can get lost / transmit can get delayed /
etc. I'm also testing through a MIPS CPU based bridge, so I'm also not
bridging at line rate. (The above is from one of the x86 laptops doing
the traffic test.) These are both reasons why I may be poking at a
path that you don't normally see. :)
I appreciate you taking a very quick look at this!
Thanks,
-adrian
>
>>
>> (kgdb) bt
>> #0 doadump (textdump=3D0) at pcpu.h:221
>> #1 0xffffffff803666b6 in db_fncall (dummy1=3D<value optimized out>,
>> dummy2=3D<value optimized out>, dummy3=3D<value optimized out>,
>> dummy4=3D<value optimized out>) at
>> /usr/home/adrian/work/freebsd/head/src/sys/ddb/db_command.c:568
>> #2 0xffffffff8036614e in db_command (cmd_table=3D0x0) at
>> /usr/home/adrian/work/freebsd/head/src/sys/ddb/db_command.c:440
>> #3 0xffffffff80365ee4 in db_command_loop () at
>> /usr/home/adrian/work/freebsd/head/src/sys/ddb/db_command.c:493
>> #4 0xffffffff8036897b in db_trap (type=3D<value optimized out>, code=3D0=
)
>> at /usr/home/adrian/work/freebsd/head/src/sys/ddb/db_main.c:251
>> #5 0xffffffff8096c0f3 in kdb_trap (type=3D9, code=3D0, tf=3D<value
>> optimized out>) at
>> /usr/home/adrian/work/freebsd/head/src/sys/kern/subr_kdb.c:654
>> #6 0xffffffff80d34c81 in trap_fatal (frame=3D0xfffffe022815d7a0,
>> eva=3D<value optimized out>) at
>> /usr/home/adrian/work/freebsd/head/src/sys/amd64/amd64/trap.c:829
>> #7 0xffffffff80d34951 in trap (frame=3D<value optimized out>) at
>> /usr/home/adrian/work/freebsd/head/src/sys/amd64/amd64/trap.c:203
>> #8 0xffffffff80d149f7 in calltrap () at
>> /usr/home/adrian/work/freebsd/head/src/sys/amd64/amd64/exception.S:234
>> #9 0xffffffff8092c3fb in _rw_wlock_cookie (c=3D0xdeadc0dedeadc2de,
>> file=3D0xffffffff81211b1f
>> "/usr/home/adrian/work/freebsd/head/src/sys/netinet/if_ether.c",
>> line=3D205)
>> at /usr/home/adrian/work/freebsd/head/src/sys/kern/kern_rwlock.c:261
>> #10 0xffffffff80a2487f in arptimer (arg=3D0xfffff8005ecc4000) at
>> /usr/home/adrian/work/freebsd/head/src/sys/netinet/if_ether.c:205
>> #11 0xffffffff80944c24 in softclock_call_cc (c=3D0xfffff8005ecc40a8,
>> cc=3D0xffffffff81b2d480, direct=3D0) at
>> /usr/home/adrian/work/freebsd/head/src/sys/kern/kern_timeout.c:722
>> #12 0xffffffff80944f87 in softclock (arg=3D<value optimized out>) at
>> /usr/home/adrian/work/freebsd/head/src/sys/kern/kern_timeout.c:851
>> #13 0xffffffff808f7eb6 in intr_event_execute_handlers (p=3D<value
>> optimized out>, ie=3D0xfffff800035a6600) at
>> /usr/home/adrian/work/freebsd/head/src/sys/kern/kern_intr.c:1262
>> #14 0xffffffff808f8546 in ithread_loop (arg=3D0xfffff800032c47c0) at
>> /usr/home/adrian/work/freebsd/head/src/sys/kern/kern_intr.c:1275
>> #15 0xffffffff808f57a4 in fork_exit (callout=3D0xffffffff808f84a0
>> <ithread_loop>, arg=3D0xfffff800032c47c0, frame=3D0xfffffe022815dac0) at
>> /usr/home/adrian/work/freebsd/head/src/sys/kern/kern_fork.c:1011
>> #16 0xffffffff80d14f2e in fork_trampoline () at
>> /usr/home/adrian/work/freebsd/head/src/sys/amd64/amd64/exception.S:609
>> #17 0x0000000000000000 in ?? ()
>> Current language: auto; currently minimal
>>
>> (kgdb) print *(struct llentry *)c_arg
>> $2 =3D {lle_next =3D {le_next =3D 0x0, le_prev =3D 0xfffff8005e867dc8},
>> r_l3addr =3D {addr4 =3D {s_addr =3D 16782508}, addr6 =3D {__u6_addr =3D
>> {__u6_addr8 =3D 0xfffff8005ecc4010 "=EF=BF=BD\024", __u6_addr16 =3D
>> 0xfffff8005ecc4010,
>> __u6_addr32 =3D 0xfffff8005ecc4010}}}, ll_addr =3D {mac_aligned =
=3D
>> 110869256150596, mac16 =3D 0xfffff8005ecc4020, mac8 =3D 0xfffff8005ecc40=
20
>> "D\036=EF=BF=BD=EF=BF=BD=EF=BF=BDd"}, spare0 =3D 0, spare1 =3D 0, lle_tb=
l =3D 0xfffff8005e867e00,
>> lle_head =3D 0xfffff8005e867dc8, lle_free =3D 0xffffffff80a2c5d0
>> <in_lltable_destroy_lle>, la_hold =3D 0x0, la_numheld =3D 0, la_expire =
=3D
>> 2110, la_flags =3D 1, la_asked =3D 0, la_preempt =3D 5, ln_state =3D 0,
>> ln_router =3D 0, ln_ntick =3D 0,
>> lle_refcnt =3D 1, lle_chain =3D {le_next =3D 0x0, le_prev =3D 0x0},
>> lle_timer =3D {c_links =3D {le =3D {le_next =3D 0x0, le_prev =3D
>> 0xffffffff81b2d588}, sle =3D {sle_next =3D 0x0}, tqe =3D {tqe_next =3D 0=
x0,
>> tqe_prev =3D 0xffffffff81b2d588}},
>> c_time =3D 9066299815445, c_precision =3D 322122525000, c_arg =3D
>> 0xfffff8005ecc4000, c_func =3D 0xffffffff80a246e0 <arptimer>, c_lock =3D
>> 0x0, c_flags =3D 0, c_iflags =3D 144, c_cpu =3D 0}, lle_lock =3D {lock_o=
bject
>> =3D {
>> lo_name =3D 0xffffffff8120fbce "lle", lo_flags =3D 90374144, lo_da=
ta
>> =3D 0, lo_witness =3D 0xfffffe0000b53c80}, rw_lock =3D 1}}
>>
>> ..
>>
>> (kgdb) print *((struct llentry *)c_arg)->lle_tbl
>> $4 =3D {llt_link =3D {sle_next =3D 0xdeadc0dedeadc0de}, llt_af =3D -5590=
38242,
>> llt_hsize =3D -559038242, lle_head =3D 0xdeadc0dedeadc0de, llt_ifp =3D
>> 0xdeadc0dedeadc0de, llt_lookup =3D 0xdeadc0dedeadc0de,
>> llt_alloc_entry =3D 0xdeadc0dedeadc0de, llt_delete_entry =3D
>> 0xdeadc0dedeadc0de, llt_prefix_free =3D 0xdeadc0dedeadc0de,
>> llt_dump_entry =3D 0xdeadc0dedeadc0de, llt_hash =3D 0xdeadc0dedeadc0de,
>> llt_match_prefix =3D 0xdeadc0dedeadc0de,
>> llt_free_entry =3D 0xdeadc0dedeadc0de, llt_foreach_entry =3D
>> 0xdeadc0dedeadc0de, llt_link_entry =3D 0xdeadc0dedeadc0de,
>> llt_unlink_entry =3D 0xdeadc0dedeadc0de, llt_fill_sa_entry =3D
>> 0xdeadc0dedeadc0de,
>> llt_free_tbl =3D 0xdeadc0dedeadc0de}
>>
>> :(
>>
>> Any ideas on where next to look?
>>
>> -adrian
>> _______________________________________________
>> freebsd-net@freebsd.org mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-net
>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJ-Vmo=JjHonDqOYK%2BJDaf9581dRU5_KoaSTnY27JnzQm0v56w>