Date: Thu, 26 Apr 2012 10:03:19 -0400 From: Ryan Stone <rysto32@gmail.com> To: "Li, Qing" <qing.li@bluecoat.com> Cc: freebsd-net <freebsd-net@freebsd.org> Subject: Re: Removing an IPv6 address does not remove NDP entries on that subnet Message-ID: <CAFMmRNyvvZEe9=jCq928pYSxNbWyy9Av9vz-vEgJoHEz1Kv26w@mail.gmail.com> In-Reply-To: <B143A8975061C446AD5E29742C531723C7FD74@pwsvl-excmbx-05.internal.cacheflow.com> References: <CAFMmRNyK6RXb43kCRxZbZPSWmmGHYx-1cxsTgL1orVjoDcKYAg@mail.gmail.com> <B143A8975061C446AD5E29742C531723C4C6F8@pwsvl-excmbx-05.internal.cacheflow.com> <CAFMmRNxWUw4XmsNZZi%2BzVjZK6i-Ukisqyub2MsOY11Nb8T9ZCQ@mail.gmail.com> <B143A8975061C446AD5E29742C531723C510D6@pwsvl-excmbx-05.internal.cacheflow.com> <B143A8975061C446AD5E29742C531723C648D5@pwsvl-excmbx-05.internal.cacheflow.com> <CAFMmRNwBt3s4a2_zYN14FQ=tC6nRR2_F4hdfRRu_Gb8SWPLYrw@mail.gmail.com> <B143A8975061C446AD5E29742C531723C64E49@pwsvl-excmbx-05.internal.cacheflow.com> <CAFMmRNw6io7Vn_8ifvLabs%2B1eW03jLBVE4=_CfztuABe6ZrbYg@mail.gmail.com> <B143A8975061C446AD5E29742C531723C6622A@pwsvl-excmbx-05.internal.cacheflow.com> <B143A8975061C446AD5E29742C531723C7FD74@pwsvl-excmbx-05.internal.cacheflow.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, Apr 25, 2012 at 3:59 AM, Li, Qing <qing.li@bluecoat.com> wrote: > The patch is located at > > =A0 http://people.freebsd.org/~qingli/nd6_prefix.diff > > Please give it a try. I did only basic testing as of now and > will do more tomorrow. > > --Qing I tested this last night. Unfortunately this seems to be vulnerable to the same races as our current ARP implementation. First, there is a race between in6_lltable_lookup and in6_lltable_prefix_free. Here is a crash that I reproduced last night: Fatal trap 12: page fault while in kernel mode cpuid =3D 0; apic id =3D 00 fault virtual address =3D 0x360 fault code =3D supervisor read data, page not present instruction pointer =3D 0x20:0xffffffff808731a2 stack pointer =3D 0x28:0xffffff80003ea280 frame pointer =3D 0x28:0xffffff80003ea320 code segment =3D base 0x0, limit 0xfffff, type 0x1b =3D DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags =3D interrupt enabled, resume, IOPL =3D 0 current process =3D 22170 (nc) trap number =3D 12 panic: page fault cpuid =3D 0 Uptime: 1h10m28s Dumping 108 out of 489 MB:..15%..30%..45%..60%..74%..89% #0 doadump (textdump=3D1) at /usr/home/rstone/freebsd/head/sys/kern/kern_shutdown.c:268 268 if (textdump && textdump_pending) { (kgdb) #0 doadump (textdump=3D1) at /usr/home/rstone/freebsd/head/sys/kern/kern_shutdown.c:268 #1 0xffffffff808753b9 in kern_reboot (howto=3D260) at /usr/home/rstone/freebsd/head/sys/kern/kern_shutdown.c:454 #2 0xffffffff80874dea in panic (fmt=3D0x0) at /usr/home/rstone/freebsd/head/sys/kern/kern_shutdown.c:642 #3 0xffffffff80b72860 in trap_fatal (frame=3D0xc, eva=3DVariable "eva" is = not avail able. ) at /usr/home/rstone/freebsd/head/sys/amd64/amd64/trap.c:852 #4 0xffffffff80b72a2a in trap_pfault (frame=3D0xffffff80003ea1d0, usermode= =3D0) at /usr/home/rstone/freebsd/head/sys/amd64/amd64/trap.c:769 #5 0xffffffff80b731ea in trap (frame=3D0xffffff80003ea1d0) at /usr/home/rstone/freebsd/head/sys/amd64/amd64/trap.c:456 #6 0xffffffff80b5d7c3 in calltrap () at /usr/home/rstone/freebsd/head/sys/amd64/amd64/exception.S:228 #7 0xffffffff808731a2 in _rw_rlock (rw=3D0xfffffe00058f3410, file=3D0xffffffff80df7e40 "/usr/home/rstone/freebsd/head/sys/netinet6/i= n6.c", line=3D2615) at /usr/home/rstone/freebsd/head/sys/kern/kern_rwlock.c:388 #8 0xffffffff80a27bd9 in in6_lltable_lookup (llt=3D0xfffffe0002422600, flags=3D0, l3addr=3D0xffffff80003ea6ec) at /usr/home/rstone/freebsd/head/sys/netinet6/in6.c:2615 #9 0xffffffff80a409c6 in nd6_storelladdr (ifp=3D0xfffffe000241f000, m=3D0xfffffe000503d500, dst=3D0xffffff80003ea6ec, desten=3D0xffffff80003ea430 "\xc0\230\b\005", lle=3D0xffffff80003ea428) #10 0xffffffff80936f24 in ether_output (ifp=3D0xfffffe000241f000, m=3D0xfffffe000503d500, dst=3D0xffffff80003ea6ec, ro=3DVariable "ro" is= not availa ble. ) at /usr/home/rstone/freebsd/head/sys/net/if_ethersubr.c:235 #11 0xffffffff80a417df in nd6_output_lle (ifp=3D0xfffffe000241f000, origifp=3D0xfffffe000241f000, m0=3D0xfffffe000503d500, dst=3D0xffffff80003ea6ec, rt0=3DVariable "rt0" is not available. ) at /usr/home/rstone/freebsd/head/sys/netinet6/nd6.c:2081 #12 0xffffffff80a41a18 in nd6_output (ifp=3DVariable "ifp" is not available= . ) at /usr/home/rstone/freebsd/head/sys/netinet6/nd6.c:1828 #13 0xffffffff80a3c400 in ip6_output (m0=3D0x0, opt=3DVariable "opt" is not available.) at /usr/home/rstone/freebsd/head/sys/netinet6/ip6_output.c:1123 #14 0xffffffff80a4debe in udp6_send (so=3DVariable "so" is not available. ) at /usr/home/rstone/freebsd/head/sys/netinet6/udp6_usrreq.c:782 #15 0xffffffff808e97de in sosend_dgram (so=3D0xfffffe00058d3d48, addr=3D0x0, uio=3DVariable "uio" is not available. at /usr/home/rstone/freebsd/head/sys/kern/uipc_socket.c:1118 #16 0xffffffff808cddad in soo_write (fp=3DVariable "fp" is not available. ) at /usr/home/rstone/freebsd/head/sys/kern/sys_socket.c:102 #17 0xffffffff808c5f25 in dofilewrite (td=3D0xfffffe00050898c0, fd=3D3, fp=3D0xfffffe0005134000, auio=3D0xffffff80003eaad0, offset=3DVariable "= offset" is not available. ) at file.h:271 #18 0xffffffff808c65ac in kern_writev (td=3D0xfffffe00050898c0, fd=3D3, auio=3D0xffffff80003eaad0) at /usr/home/rstone/freebsd/head/sys/kern/sys_generic.c:459 #19 0xffffffff808c66c4 in sys_write (td=3DVariable "td" is not available. ) at /usr/home/rstone/freebsd/head/sys/kern/sys_generic.c:375 #20 0xffffffff80b71f79 in amd64_syscall (td=3D0xfffffe00050898c0, traced=3D= 0) at subr_syscall.c:135 #21 0xffffffff80b5daa7 in Xfast_syscall () at /usr/home/rstone/freebsd/head/sys/amd64/amd64/exception.S:387 The line in in6_lltable_lookup that it crashes at (in6.c:2615) is the LLE_RLOCK at the end of the function. From the vmcore I can see that the lle was destroyed: (kgdb) print *lle $1 =3D {lle_next =3D {le_next =3D 0xdeadc0dedeadc0de, le_prev =3D 0xdeadc0d= edeadc0de}, lle_lock =3D {lock_object =3D { lo_name =3D 0xdeadc0dedeadc0de <Address 0xdeadc0dedeadc0de out of bou= nds>, lo_flags =3D 3735929054, lo_data =3D 3735929054, lo_witness =3D 0xdeadc0dedeadc0de}, rw_lock =3D 16045693110842147038}= , lle_tbl =3D 0xdeadc0dedeadc0de, lle_head =3D 0xdeadc0dedeadc0de, lle_free =3D 0xdeadc0dedeadc0de, la_hold =3D 0xdeadc0dedeadc0de, la_numheld =3D -559038242, la_expire =3D -2401050962867404578, la_flags = =3D 49374, la_asked =3D 57005, la_preempt =3D 49374, ln_byhint =3D 57005, ln_state = =3D -16162, ln_router =3D 57005, ln_ntick =3D -2401050962867404578, lle_refcnt =3D -5= 59038242, ll_addr =3D {mac_aligned =3D 16045693110842147038, mac16 =3D {49374, 5700= 5, 49374}}, lle_timer =3D {ln_timer_ch =3D {c_links =3D {sle =3D { sle_next =3D 0xdeadc0dedeadc0de}, tqe =3D { tqe_next =3D 0xdeadc0dedeadc0de, tqe_prev =3D 0xdeadc0dedeadc0de}= }, c_time =3D -559038242, c_arg =3D 0xdeadc0dedeadc0de, c_func =3D 0xdeadc0dedeadc0de, c_lock =3D 0xdeadc0dedeadc0de, c_flags =3D -559038242, c_cpu =3D -559038242}, la_timer =3D {c_links = =3D {sle =3D { sle_next =3D 0xdeadc0dedeadc0de}, tqe =3D { tqe_next =3D 0xdeadc0dedeadc0de, tqe_prev =3D 0xdeadc0dedeadc0de}= }, c_time =3D -559038242, c_arg =3D 0xdeadc0dedeadc0de, c_func =3D 0xdeadc0dedeadc0de, c_lock =3D 0xdeadc0dedeadc0de, c_flags =3D -559038242, c_cpu =3D -559038242}}} The test that I was running was I had one ping6 -f going to an IPv6 address as well as a loop of nc -u. In another terminal I had a script constantly removing and adding the ipv6 address from which I was pinging/ncing. I haven't looked at the netinet6 code too closely yet but if it follows the netinet implementation in6_lltable_prefix_free should acquire the afdata_lock on the ifp before touching the lltable. I haven't tried a test for this yet, but I also believe that in6_lltable_prefix_free also doesn't drain the callout in the llentry correctly. I try testing this to confirm this now.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAFMmRNyvvZEe9=jCq928pYSxNbWyy9Av9vz-vEgJoHEz1Kv26w>