Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 26 Apr 2012 10:03:19 -0400
From:      Ryan Stone <rysto32@gmail.com>
To:        "Li, Qing" <qing.li@bluecoat.com>
Cc:        freebsd-net <freebsd-net@freebsd.org>
Subject:   Re: Removing an IPv6 address does not remove NDP entries on that subnet
Message-ID:  <CAFMmRNyvvZEe9=jCq928pYSxNbWyy9Av9vz-vEgJoHEz1Kv26w@mail.gmail.com>
In-Reply-To: <B143A8975061C446AD5E29742C531723C7FD74@pwsvl-excmbx-05.internal.cacheflow.com>
References:  <CAFMmRNyK6RXb43kCRxZbZPSWmmGHYx-1cxsTgL1orVjoDcKYAg@mail.gmail.com> <B143A8975061C446AD5E29742C531723C4C6F8@pwsvl-excmbx-05.internal.cacheflow.com> <CAFMmRNxWUw4XmsNZZi%2BzVjZK6i-Ukisqyub2MsOY11Nb8T9ZCQ@mail.gmail.com> <B143A8975061C446AD5E29742C531723C510D6@pwsvl-excmbx-05.internal.cacheflow.com> <B143A8975061C446AD5E29742C531723C648D5@pwsvl-excmbx-05.internal.cacheflow.com> <CAFMmRNwBt3s4a2_zYN14FQ=tC6nRR2_F4hdfRRu_Gb8SWPLYrw@mail.gmail.com> <B143A8975061C446AD5E29742C531723C64E49@pwsvl-excmbx-05.internal.cacheflow.com> <CAFMmRNw6io7Vn_8ifvLabs%2B1eW03jLBVE4=_CfztuABe6ZrbYg@mail.gmail.com> <B143A8975061C446AD5E29742C531723C6622A@pwsvl-excmbx-05.internal.cacheflow.com> <B143A8975061C446AD5E29742C531723C7FD74@pwsvl-excmbx-05.internal.cacheflow.com>

next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, Apr 25, 2012 at 3:59 AM, Li, Qing <qing.li@bluecoat.com> wrote:
> The patch is located at
>
> =A0 http://people.freebsd.org/~qingli/nd6_prefix.diff
>
> Please give it a try. I did only basic testing as of now and
> will do more tomorrow.
>
> --Qing

I tested this last night.  Unfortunately this seems to be vulnerable
to the same races as our current ARP implementation.  First, there is
a race between in6_lltable_lookup and in6_lltable_prefix_free.  Here
is a crash that I reproduced last night:

Fatal trap 12: page fault while in kernel mode
cpuid =3D 0; apic id =3D 00
fault virtual address   =3D 0x360
fault code              =3D supervisor read data, page not present
instruction pointer     =3D 0x20:0xffffffff808731a2
stack pointer           =3D 0x28:0xffffff80003ea280
frame pointer           =3D 0x28:0xffffff80003ea320
code segment            =3D base 0x0, limit 0xfffff, type 0x1b
                        =3D DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags        =3D interrupt enabled, resume, IOPL =3D 0
current process         =3D 22170 (nc)
trap number             =3D 12
panic: page fault
cpuid =3D 0
Uptime: 1h10m28s
Dumping 108 out of 489 MB:..15%..30%..45%..60%..74%..89%

#0  doadump (textdump=3D1)
    at /usr/home/rstone/freebsd/head/sys/kern/kern_shutdown.c:268
268             if (textdump && textdump_pending) {
(kgdb) #0  doadump (textdump=3D1)
    at /usr/home/rstone/freebsd/head/sys/kern/kern_shutdown.c:268
#1  0xffffffff808753b9 in kern_reboot (howto=3D260)
    at /usr/home/rstone/freebsd/head/sys/kern/kern_shutdown.c:454
#2  0xffffffff80874dea in panic (fmt=3D0x0)
    at /usr/home/rstone/freebsd/head/sys/kern/kern_shutdown.c:642
#3  0xffffffff80b72860 in trap_fatal (frame=3D0xc, eva=3DVariable "eva" is =
not avail
able.
)
    at /usr/home/rstone/freebsd/head/sys/amd64/amd64/trap.c:852
#4  0xffffffff80b72a2a in trap_pfault (frame=3D0xffffff80003ea1d0, usermode=
=3D0)
    at /usr/home/rstone/freebsd/head/sys/amd64/amd64/trap.c:769
#5  0xffffffff80b731ea in trap (frame=3D0xffffff80003ea1d0)
    at /usr/home/rstone/freebsd/head/sys/amd64/amd64/trap.c:456
#6  0xffffffff80b5d7c3 in calltrap ()
    at /usr/home/rstone/freebsd/head/sys/amd64/amd64/exception.S:228
#7  0xffffffff808731a2 in _rw_rlock (rw=3D0xfffffe00058f3410,
    file=3D0xffffffff80df7e40 "/usr/home/rstone/freebsd/head/sys/netinet6/i=
n6.c",
line=3D2615) at /usr/home/rstone/freebsd/head/sys/kern/kern_rwlock.c:388
#8  0xffffffff80a27bd9 in in6_lltable_lookup (llt=3D0xfffffe0002422600,
    flags=3D0, l3addr=3D0xffffff80003ea6ec)
    at /usr/home/rstone/freebsd/head/sys/netinet6/in6.c:2615
#9  0xffffffff80a409c6 in nd6_storelladdr (ifp=3D0xfffffe000241f000,
    m=3D0xfffffe000503d500, dst=3D0xffffff80003ea6ec,
    desten=3D0xffffff80003ea430 "\xc0\230\b\005", lle=3D0xffffff80003ea428)
#10 0xffffffff80936f24 in ether_output (ifp=3D0xfffffe000241f000,
    m=3D0xfffffe000503d500, dst=3D0xffffff80003ea6ec, ro=3DVariable "ro" is=
 not availa
ble.
)
    at /usr/home/rstone/freebsd/head/sys/net/if_ethersubr.c:235
#11 0xffffffff80a417df in nd6_output_lle (ifp=3D0xfffffe000241f000,
    origifp=3D0xfffffe000241f000, m0=3D0xfffffe000503d500,
    dst=3D0xffffff80003ea6ec, rt0=3DVariable "rt0" is not available.
)
    at /usr/home/rstone/freebsd/head/sys/netinet6/nd6.c:2081
#12 0xffffffff80a41a18 in nd6_output (ifp=3DVariable "ifp" is not available=
.
)
    at /usr/home/rstone/freebsd/head/sys/netinet6/nd6.c:1828
#13 0xffffffff80a3c400 in ip6_output (m0=3D0x0, opt=3DVariable "opt" is
not available.)
    at /usr/home/rstone/freebsd/head/sys/netinet6/ip6_output.c:1123
#14 0xffffffff80a4debe in udp6_send (so=3DVariable "so" is not available.
)
    at /usr/home/rstone/freebsd/head/sys/netinet6/udp6_usrreq.c:782
#15 0xffffffff808e97de in sosend_dgram (so=3D0xfffffe00058d3d48,
addr=3D0x0, uio=3DVariable "uio" is not available.
    at /usr/home/rstone/freebsd/head/sys/kern/uipc_socket.c:1118
#16 0xffffffff808cddad in soo_write (fp=3DVariable "fp" is not available.
)
    at /usr/home/rstone/freebsd/head/sys/kern/sys_socket.c:102
#17 0xffffffff808c5f25 in dofilewrite (td=3D0xfffffe00050898c0, fd=3D3,
    fp=3D0xfffffe0005134000, auio=3D0xffffff80003eaad0, offset=3DVariable "=
offset" is
not available.
) at file.h:271
#18 0xffffffff808c65ac in kern_writev (td=3D0xfffffe00050898c0, fd=3D3,
    auio=3D0xffffff80003eaad0)
    at /usr/home/rstone/freebsd/head/sys/kern/sys_generic.c:459
#19 0xffffffff808c66c4 in sys_write (td=3DVariable "td" is not available.
)
    at /usr/home/rstone/freebsd/head/sys/kern/sys_generic.c:375
#20 0xffffffff80b71f79 in amd64_syscall (td=3D0xfffffe00050898c0, traced=3D=
0)
    at subr_syscall.c:135
#21 0xffffffff80b5daa7 in Xfast_syscall ()
    at /usr/home/rstone/freebsd/head/sys/amd64/amd64/exception.S:387

The line in in6_lltable_lookup that it crashes at (in6.c:2615) is the
LLE_RLOCK at the end of the function.  From the vmcore I can see that
the lle was destroyed:

(kgdb) print *lle
$1 =3D {lle_next =3D {le_next =3D 0xdeadc0dedeadc0de, le_prev =3D 0xdeadc0d=
edeadc0de},
  lle_lock =3D {lock_object =3D {
      lo_name =3D 0xdeadc0dedeadc0de <Address 0xdeadc0dedeadc0de out of bou=
nds>,
      lo_flags =3D 3735929054, lo_data =3D 3735929054,
      lo_witness =3D 0xdeadc0dedeadc0de}, rw_lock =3D 16045693110842147038}=
,
  lle_tbl =3D 0xdeadc0dedeadc0de, lle_head =3D 0xdeadc0dedeadc0de,
  lle_free =3D 0xdeadc0dedeadc0de, la_hold =3D 0xdeadc0dedeadc0de,
  la_numheld =3D -559038242, la_expire =3D -2401050962867404578, la_flags =
=3D 49374,
  la_asked =3D 57005, la_preempt =3D 49374, ln_byhint =3D 57005, ln_state =
=3D -16162,
  ln_router =3D 57005, ln_ntick =3D -2401050962867404578, lle_refcnt =3D -5=
59038242,
  ll_addr =3D {mac_aligned =3D 16045693110842147038, mac16 =3D {49374, 5700=
5,
      49374}}, lle_timer =3D {ln_timer_ch =3D {c_links =3D {sle =3D {
          sle_next =3D 0xdeadc0dedeadc0de}, tqe =3D {
          tqe_next =3D 0xdeadc0dedeadc0de, tqe_prev =3D 0xdeadc0dedeadc0de}=
},
      c_time =3D -559038242, c_arg =3D 0xdeadc0dedeadc0de,
      c_func =3D 0xdeadc0dedeadc0de, c_lock =3D 0xdeadc0dedeadc0de,
      c_flags =3D -559038242, c_cpu =3D -559038242}, la_timer =3D {c_links =
=3D {sle =3D {
          sle_next =3D 0xdeadc0dedeadc0de}, tqe =3D {
          tqe_next =3D 0xdeadc0dedeadc0de, tqe_prev =3D 0xdeadc0dedeadc0de}=
},
      c_time =3D -559038242, c_arg =3D 0xdeadc0dedeadc0de,
      c_func =3D 0xdeadc0dedeadc0de, c_lock =3D 0xdeadc0dedeadc0de,
      c_flags =3D -559038242, c_cpu =3D -559038242}}}


The test that I was running was I had one ping6 -f going to an IPv6
address as well as a loop of nc -u.  In another terminal I had a
script constantly removing and adding the ipv6 address from which I
was pinging/ncing.

I haven't looked at the netinet6 code too closely yet but if it
follows the netinet implementation in6_lltable_prefix_free should
acquire the afdata_lock on the ifp before touching the lltable.


I haven't tried a test for this yet, but I also believe that
in6_lltable_prefix_free also doesn't drain the callout in the llentry
correctly.  I try testing this to confirm this now.



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAFMmRNyvvZEe9=jCq928pYSxNbWyy9Av9vz-vEgJoHEz1Kv26w>