From owner-freebsd-net@freebsd.org Sat Oct 31 15:01:22 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A947BA22DD9; Sat, 31 Oct 2015 15:01:22 +0000 (UTC) (envelope-from melifaro@ipfw.ru) Received: from forward11h.cmail.yandex.net (forward11h.cmail.yandex.net [87.250.230.153]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "forwards.mail.yandex.net", Issuer "Certum Level IV CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 42BF81BAA; Sat, 31 Oct 2015 15:01:21 +0000 (UTC) (envelope-from melifaro@ipfw.ru) Received: from web19h.yandex.ru (web19h.yandex.ru [84.201.186.48]) by forward11h.cmail.yandex.net (Yandex) with ESMTP id 4BD5C20F8A; Sat, 31 Oct 2015 18:01:17 +0300 (MSK) Received: from 127.0.0.1 (localhost [127.0.0.1]) by web19h.yandex.ru (Yandex) with ESMTP id 9E9B12E011BF; Sat, 31 Oct 2015 18:01:16 +0300 (MSK) Received: by web19h.yandex.ru with HTTP; Sat, 31 Oct 2015 18:01:15 +0300 From: Alexander V. Chernikov Envelope-From: melifaro@ipfw.ru To: Adrian Chadd Cc: freebsd-current , FreeBSD Net In-Reply-To: References: <2739461446298483@web2h.yandex.ru> Subject: Re: panic in arptimer in r289937 MIME-Version: 1.0 Message-Id: <1733241446303675@web19h.yandex.ru> X-Mailer: Yamail [ http://yandex.ru ] 5.0 Date: Sat, 31 Oct 2015 18:01:15 +0300 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=utf-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 31 Oct 2015 15:01:22 -0000 31.10.2015, 16:46, "Adrian Chadd" : > On 31 October 2015 at 09:34, Alexander V. Chernikov > wrote: >>  31.10.2015, 05:32, "Adrian Chadd" : >>>  Hiya, >>> >>>  Here's a panic from arptimer: >>  Hi Adrian, >> >>  As far as I see, line 205 in if_ether.c is IF_AFDATA_LOCK(ifp) which happens after LLE_WUNLOCK(). >>  So, it looks like (pre-cached) ifp had been freed before locking ifdata. >>  Do you have any more details on that? (e.g. was some interface detached at that moment, is it reproducible, etc..) >> >>  From a quick glance, potential use-after-free has been possible for quite a long time, but I wonder why it hasn't been observed before. >>  Probably lltable_free() changes might have triggered that. >> >>  I'll take a deeper look on that and reply. > > Hiya! > > Thanks for your quick response. > > I mean, I use wifi, and ARPs can get lost / transmit can get delayed / > etc. I'm also testing through a MIPS CPU based bridge, so I'm also not I remember that :) > bridging at line rate. (The above is from one of the x86 laptops doing > the traffic test.) These are both reasons why I may be poking at a > path that you don't normally see. :) Yup. So, once again, could you provide a bit more details? :) Was it related with any interface being destroyed? What was the test scenario (just bridging between interfaces?) Is this reproducible? > > I appreciate you taking a very quick look at this! > > Thanks, > > -adrian > >>>  (kgdb) bt >>>  #0 doadump (textdump=0) at pcpu.h:221 >>>  #1 0xffffffff803666b6 in db_fncall (dummy1=, >>>  dummy2=, dummy3=, >>>  dummy4=) at >>>  /usr/home/adrian/work/freebsd/head/src/sys/ddb/db_command.c:568 >>>  #2 0xffffffff8036614e in db_command (cmd_table=0x0) at >>>  /usr/home/adrian/work/freebsd/head/src/sys/ddb/db_command.c:440 >>>  #3 0xffffffff80365ee4 in db_command_loop () at >>>  /usr/home/adrian/work/freebsd/head/src/sys/ddb/db_command.c:493 >>>  #4 0xffffffff8036897b in db_trap (type=, code=0) >>>  at /usr/home/adrian/work/freebsd/head/src/sys/ddb/db_main.c:251 >>>  #5 0xffffffff8096c0f3 in kdb_trap (type=9, code=0, tf=>>  optimized out>) at >>>  /usr/home/adrian/work/freebsd/head/src/sys/kern/subr_kdb.c:654 >>>  #6 0xffffffff80d34c81 in trap_fatal (frame=0xfffffe022815d7a0, >>>  eva=) at >>>  /usr/home/adrian/work/freebsd/head/src/sys/amd64/amd64/trap.c:829 >>>  #7 0xffffffff80d34951 in trap (frame=) at >>>  /usr/home/adrian/work/freebsd/head/src/sys/amd64/amd64/trap.c:203 >>>  #8 0xffffffff80d149f7 in calltrap () at >>>  /usr/home/adrian/work/freebsd/head/src/sys/amd64/amd64/exception.S:234 >>>  #9 0xffffffff8092c3fb in _rw_wlock_cookie (c=0xdeadc0dedeadc2de, >>>  file=0xffffffff81211b1f >>>  "/usr/home/adrian/work/freebsd/head/src/sys/netinet/if_ether.c", >>>  line=205) >>>      at /usr/home/adrian/work/freebsd/head/src/sys/kern/kern_rwlock.c:261 >>>  #10 0xffffffff80a2487f in arptimer (arg=0xfffff8005ecc4000) at >>>  /usr/home/adrian/work/freebsd/head/src/sys/netinet/if_ether.c:205 >>>  #11 0xffffffff80944c24 in softclock_call_cc (c=0xfffff8005ecc40a8, >>>  cc=0xffffffff81b2d480, direct=0) at >>>  /usr/home/adrian/work/freebsd/head/src/sys/kern/kern_timeout.c:722 >>>  #12 0xffffffff80944f87 in softclock (arg=) at >>>  /usr/home/adrian/work/freebsd/head/src/sys/kern/kern_timeout.c:851 >>>  #13 0xffffffff808f7eb6 in intr_event_execute_handlers (p=>>  optimized out>, ie=0xfffff800035a6600) at >>>  /usr/home/adrian/work/freebsd/head/src/sys/kern/kern_intr.c:1262 >>>  #14 0xffffffff808f8546 in ithread_loop (arg=0xfffff800032c47c0) at >>>  /usr/home/adrian/work/freebsd/head/src/sys/kern/kern_intr.c:1275 >>>  #15 0xffffffff808f57a4 in fork_exit (callout=0xffffffff808f84a0 >>>  , arg=0xfffff800032c47c0, frame=0xfffffe022815dac0) at >>>  /usr/home/adrian/work/freebsd/head/src/sys/kern/kern_fork.c:1011 >>>  #16 0xffffffff80d14f2e in fork_trampoline () at >>>  /usr/home/adrian/work/freebsd/head/src/sys/amd64/amd64/exception.S:609 >>>  #17 0x0000000000000000 in ?? () >>>  Current language: auto; currently minimal >>> >>>  (kgdb) print *(struct llentry *)c_arg >>>  $2 = {lle_next = {le_next = 0x0, le_prev = 0xfffff8005e867dc8}, >>>  r_l3addr = {addr4 = {s_addr = 16782508}, addr6 = {__u6_addr = >>>  {__u6_addr8 = 0xfffff8005ecc4010 "�\024", __u6_addr16 = >>>  0xfffff8005ecc4010, >>>          __u6_addr32 = 0xfffff8005ecc4010}}}, ll_addr = {mac_aligned = >>>  110869256150596, mac16 = 0xfffff8005ecc4020, mac8 = 0xfffff8005ecc4020 >>>  "D\036���d"}, spare0 = 0, spare1 = 0, lle_tbl = 0xfffff8005e867e00, >>>    lle_head = 0xfffff8005e867dc8, lle_free = 0xffffffff80a2c5d0 >>>  , la_hold = 0x0, la_numheld = 0, la_expire = >>>  2110, la_flags = 1, la_asked = 0, la_preempt = 5, ln_state = 0, >>>  ln_router = 0, ln_ntick = 0, >>>    lle_refcnt = 1, lle_chain = {le_next = 0x0, le_prev = 0x0}, >>>  lle_timer = {c_links = {le = {le_next = 0x0, le_prev = >>>  0xffffffff81b2d588}, sle = {sle_next = 0x0}, tqe = {tqe_next = 0x0, >>>  tqe_prev = 0xffffffff81b2d588}}, >>>      c_time = 9066299815445, c_precision = 322122525000, c_arg = >>>  0xfffff8005ecc4000, c_func = 0xffffffff80a246e0 , c_lock = >>>  0x0, c_flags = 0, c_iflags = 144, c_cpu = 0}, lle_lock = {lock_object >>>  = { >>>        lo_name = 0xffffffff8120fbce "lle", lo_flags = 90374144, lo_data >>>  = 0, lo_witness = 0xfffffe0000b53c80}, rw_lock = 1}} >>> >>>  .. >>> >>>  (kgdb) print *((struct llentry *)c_arg)->lle_tbl >>>  $4 = {llt_link = {sle_next = 0xdeadc0dedeadc0de}, llt_af = -559038242, >>>  llt_hsize = -559038242, lle_head = 0xdeadc0dedeadc0de, llt_ifp = >>>  0xdeadc0dedeadc0de, llt_lookup = 0xdeadc0dedeadc0de, >>>    llt_alloc_entry = 0xdeadc0dedeadc0de, llt_delete_entry = >>>  0xdeadc0dedeadc0de, llt_prefix_free = 0xdeadc0dedeadc0de, >>>  llt_dump_entry = 0xdeadc0dedeadc0de, llt_hash = 0xdeadc0dedeadc0de, >>>  llt_match_prefix = 0xdeadc0dedeadc0de, >>>    llt_free_entry = 0xdeadc0dedeadc0de, llt_foreach_entry = >>>  0xdeadc0dedeadc0de, llt_link_entry = 0xdeadc0dedeadc0de, >>>  llt_unlink_entry = 0xdeadc0dedeadc0de, llt_fill_sa_entry = >>>  0xdeadc0dedeadc0de, >>>    llt_free_tbl = 0xdeadc0dedeadc0de} >>> >>>  :( >>> >>>  Any ideas on where next to look? >>> >>>  -adrian >>>  _______________________________________________ >>>  freebsd-net@freebsd.org mailing list >>>  https://lists.freebsd.org/mailman/listinfo/freebsd-net >>>  To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"