From owner-svn-src-stable@freebsd.org Mon Apr 17 20:13:22 2017 Return-Path: Delivered-To: svn-src-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B5B6DD42C26; Mon, 17 Apr 2017 20:13:22 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 818EEF23; Mon, 17 Apr 2017 20:13:22 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v3HKDL6j089736; Mon, 17 Apr 2017 20:13:21 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v3HKDLrr089727; Mon, 17 Apr 2017 20:13:21 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201704172013.v3HKDLrr089727@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Mon, 17 Apr 2017 20:13:21 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r317067 - in stable/11: sys/netinet6 tests/sys/netinet X-SVN-Group: stable-11 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for all the -stable branches of the src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Apr 2017 20:13:22 -0000 Author: asomers Date: Mon Apr 17 20:13:20 2017 New Revision: 317067 URL: https://svnweb.freebsd.org/changeset/base/317067 Log: MFC r313025, r313395, r314113, r314442, r315458, r315656 r313025: Add tests for multi-fib IPv6 routing PR: 196361 Submitted by: jhujhiti@adjectivism.org Reported by: Jason Healy MFC after: 4 weeks Sponsored by: Spectra Logic Corp r313395: Add fibs_test:udp_dontroute6, another IPv6 multi-FIB test PR: 196361 MFC after: 3 weeks Sponsored by: Spectra Logic Corp r314113: Remove tests/sys/netinet/fibs_tests's dependency on net/socat Instead of bridging two tap interfaces with socat, just use an epair pair. MFC after: 3 weeks Sponsored by: Spectra Logic Corp r314442: Add an ATF test for IPv6 SLAAC with multiple fibs Tests that an interface can get a SLAAC address and that it inserts its routes into the correct fib. Does not test anything to do with NDP. PR: 196361 Reviewed by: Erick Turnquist MFC after: 3 weeks Sponsored by: Spectra Logic Corp Differential Revision: https://reviews.freebsd.org/D9776 r315458: Constrain IPv6 routes to single FIBs when net.add_addr_allfibs=0 sys/netinet6/icmp6.c Use the interface's FIB for source address selection in ICMPv6 error responses. sys/netinet6/in6.c In in6_newaddrmsg, announce arrival of local addresses on the interface's FIB only. In in6_lltable_rtcheck, use a per-fib ND6 cache instead of a single cache. sys/netinet6/in6_src.c In in6_selectsrc, use the caller's fib instead of the default fib. In in6_selectsrc_socket, remove a superfluous check. sys/netinet6/nd6.c In nd6_lle_event, use the interface's fib for routing socket messages. In nd6_is_new_addr_neighbor, check all FIBs when trying to determine whether an address is a neighbor. Also, simplify the code for point to point interfaces. sys/netinet6/nd6.h sys/netinet6/nd6.c sys/netinet6/nd6_rtr.c Make defrouter_select fib-aware, and make all of its callers pass in the interface fib. sys/netinet6/nd6_nbr.c When inputting a Neighbor Solicitation packet, consider the interface fib instead of the default fib for DAD. Output NS and Neighbor Advertisement packets on the correct fib. sys/netinet6/nd6_rtr.c Allow installing the same host route on different interfaces in different FIBs. If rt_add_addr_allfibs=0, only install or delete the prefix route on the interface fib. tests/sys/netinet/fibs_test.sh Clear some expected failures, but add a skip for the newly revealed BUG217871. PR: 196361 Submitted by: Erick Turnquist Reported by: Jason Healy Reviewed by: asomers MFC after: 3 weeks Sponsored by: Spectra Logic Corp Differential Revision: https://reviews.freebsd.org/D9451 r315656: Fix back-to-back runs of sys/netinet/fibs_test;slaac_on_nondefault_fib6 This test was failing if run twice because rtadvd takes too long to die. The rtadvd process from the first run was still running when the second run created its interfaces. The solution is to use SIGKILL during the cleanup instead of SIGTERM so rtadvd will die faster. While I'm here, randomize the addresses used for the test, which makes bugs like this easier to spot, and fix the cleanup order to be the opposite of the setup order PR: 217871 MFC after: 18 days X-MFC-With: 315458 Sponsored by: Spectra Logic Corp Modified: stable/11/sys/netinet6/icmp6.c stable/11/sys/netinet6/in6.c stable/11/sys/netinet6/in6_src.c stable/11/sys/netinet6/nd6.c stable/11/sys/netinet6/nd6.h stable/11/sys/netinet6/nd6_nbr.c stable/11/sys/netinet6/nd6_rtr.c stable/11/tests/sys/netinet/fibs_test.sh stable/11/tests/sys/netinet/udp_dontroute.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/netinet6/icmp6.c ============================================================================== --- stable/11/sys/netinet6/icmp6.c Mon Apr 17 19:10:12 2017 (r317066) +++ stable/11/sys/netinet6/icmp6.c Mon Apr 17 20:13:20 2017 (r317067) @@ -2147,7 +2147,7 @@ icmp6_reflect(struct mbuf *m, size_t off * source address of the erroneous packet. */ in6_splitscope(&ip6->ip6_src, &dst6, &scopeid); - error = in6_selectsrc_addr(RT_DEFAULT_FIB, &dst6, + error = in6_selectsrc_addr(M_GETFIB(m), &dst6, scopeid, NULL, &src6, &hlim); if (error) { @@ -2289,7 +2289,7 @@ icmp6_redirect_input(struct mbuf *m, int uint32_t scopeid; in6_splitscope(&reddst6, &kdst, &scopeid); - if (fib6_lookup_nh_basic(RT_DEFAULT_FIB, &kdst, scopeid, 0, 0,&nh6)==0){ + if (fib6_lookup_nh_basic(ifp->if_fib, &kdst, scopeid, 0, 0,&nh6)==0){ if ((nh6.nh_flags & NHF_GATEWAY) == 0) { nd6log((LOG_ERR, "ICMP6 redirect rejected; no route " Modified: stable/11/sys/netinet6/in6.c ============================================================================== --- stable/11/sys/netinet6/in6.c Mon Apr 17 19:10:12 2017 (r317066) +++ stable/11/sys/netinet6/in6.c Mon Apr 17 20:13:20 2017 (r317067) @@ -159,6 +159,7 @@ in6_newaddrmsg(struct in6_ifaddr *ia, in struct sockaddr_dl gateway; struct sockaddr_in6 mask, addr; struct rtentry rt; + int fibnum; /* * initialize for rtmsg generation @@ -176,8 +177,9 @@ in6_newaddrmsg(struct in6_ifaddr *ia, in rt.rt_flags = RTF_HOST | RTF_STATIC; if (cmd == RTM_ADD) rt.rt_flags |= RTF_UP; - /* Announce arrival of local address to all FIBs. */ - rt_newaddrmsg(cmd, &ia->ia_ifa, 0, &rt); + fibnum = V_rt_add_addr_allfibs ? RT_ALL_FIBS : ia62ifa(ia)->ifa_ifp->if_fib; + /* Announce arrival of local address to this FIB. */ + rt_newaddrmsg_fib(cmd, &ia->ia_ifa, 0, &rt, fibnum); } int @@ -2115,15 +2117,15 @@ in6_lltable_rtcheck(struct ifnet *ifp, uint32_t scopeid; int error; char ip6buf[INET6_ADDRSTRLEN]; + int fibnum; KASSERT(l3addr->sa_family == AF_INET6, ("sin_family %d", l3addr->sa_family)); - /* Our local addresses are always only installed on the default FIB. */ - sin6 = (const struct sockaddr_in6 *)l3addr; in6_splitscope(&sin6->sin6_addr, &dst, &scopeid); - error = fib6_lookup_nh_basic(RT_DEFAULT_FIB, &dst, scopeid, 0, 0, &nh6); + fibnum = V_rt_add_addr_allfibs ? RT_DEFAULT_FIB : ifp->if_fib; + error = fib6_lookup_nh_basic(fibnum, &dst, scopeid, 0, 0, &nh6); if (error != 0 || (nh6.nh_flags & NHF_GATEWAY) || nh6.nh_ifp != ifp) { struct ifaddr *ifa; /* Modified: stable/11/sys/netinet6/in6_src.c ============================================================================== --- stable/11/sys/netinet6/in6_src.c Mon Apr 17 19:10:12 2017 (r317066) +++ stable/11/sys/netinet6/in6_src.c Mon Apr 17 20:13:20 2017 (r317067) @@ -297,7 +297,7 @@ in6_selectsrc(uint32_t fibnum, struct so */ /* get the outgoing interface */ if ((error = in6_selectif(dstsock, opts, mopts, &ifp, oifp, - (inp != NULL) ? inp->inp_inc.inc_fibnum : RT_DEFAULT_FIB)) != 0) + (inp != NULL) ? inp->inp_inc.inc_fibnum : fibnum)) != 0) return (error); #ifdef DIAGNOSTIC @@ -563,7 +563,7 @@ in6_selectsrc_socket(struct sockaddr_in6 uint32_t fibnum; int error; - fibnum = (inp != NULL) ? inp->inp_inc.inc_fibnum : RT_DEFAULT_FIB; + fibnum = inp->inp_inc.inc_fibnum; retifp = NULL; error = in6_selectsrc(fibnum, dstsock, opts, inp, cred, &retifp, srcp); Modified: stable/11/sys/netinet6/nd6.c ============================================================================== --- stable/11/sys/netinet6/nd6.c Mon Apr 17 19:10:12 2017 (r317066) +++ stable/11/sys/netinet6/nd6.c Mon Apr 17 20:13:20 2017 (r317067) @@ -157,6 +157,7 @@ nd6_lle_event(void *arg __unused, struct struct sockaddr_dl gw; struct ifnet *ifp; int type; + int fibnum; LLE_WLOCK_ASSERT(lle); @@ -194,8 +195,9 @@ nd6_lle_event(void *arg __unused, struct rtinfo.rti_info[RTAX_DST] = (struct sockaddr *)&dst; rtinfo.rti_info[RTAX_GATEWAY] = (struct sockaddr *)&gw; rtinfo.rti_addrs = RTA_DST | RTA_GATEWAY; + fibnum = V_rt_add_addr_allfibs ? RT_ALL_FIBS : ifp->if_fib; rt_missmsg_fib(type, &rtinfo, RTF_HOST | RTF_LLDATA | ( - type == RTM_ADD ? RTF_UP: 0), 0, RT_DEFAULT_FIB); + type == RTM_ADD ? RTF_UP: 0), 0, fibnum); } /* @@ -1200,7 +1202,7 @@ nd6_purge(struct ifnet *ifp) if (ND_IFINFO(ifp)->flags & ND6_IFF_ACCEPT_RTADV) { /* Refresh default router list. */ - defrouter_select(); + defrouter_select_fib(ifp->if_fib); } } @@ -1253,7 +1255,7 @@ static int nd6_is_new_addr_neighbor(const struct sockaddr_in6 *addr, struct ifnet *ifp) { struct nd_prefix *pr; - struct ifaddr *dstaddr; + struct ifaddr *ifa; struct rt_addrinfo info; struct sockaddr_in6 rt_key; const struct sockaddr *dst6; @@ -1287,9 +1289,6 @@ nd6_is_new_addr_neighbor(const struct so bzero(&info, sizeof(info)); info.rti_info[RTAX_DST] = (struct sockaddr *)&rt_key; - /* Always use the default FIB here. XXME - why? */ - fibnum = RT_DEFAULT_FIB; - /* * If the address matches one of our addresses, * it should be a neighbor. @@ -1303,19 +1302,31 @@ restart: continue; if ((pr->ndpr_stateflags & NDPRF_ONLINK) == 0) { - /* Always use the default FIB here. */ dst6 = (const struct sockaddr *)&pr->ndpr_prefix; - genid = V_nd6_list_genid; - ND6_RUNLOCK(); - - /* Restore length field before retrying lookup */ - rt_key.sin6_len = sizeof(rt_key); - error = rib_lookup_info(fibnum, dst6, 0, 0, &info); + /* + * We only need to check all FIBs if add_addr_allfibs + * is unset. If set, checking any FIB will suffice. + */ + fibnum = V_rt_add_addr_allfibs ? rt_numfibs - 1 : 0; + for (; fibnum < rt_numfibs; fibnum++) { + genid = V_nd6_list_genid; + ND6_RUNLOCK(); - ND6_RLOCK(); - if (genid != V_nd6_list_genid) - goto restart; + /* + * Restore length field before + * retrying lookup + */ + rt_key.sin6_len = sizeof(rt_key); + error = rib_lookup_info(fibnum, dst6, 0, 0, + &info); + + ND6_RLOCK(); + if (genid != V_nd6_list_genid) + goto restart; + if (error == 0) + break; + } if (error != 0) continue; @@ -1346,13 +1357,18 @@ restart: * If the address is assigned on the node of the other side of * a p2p interface, the address should be a neighbor. */ - dstaddr = ifa_ifwithdstaddr((const struct sockaddr *)addr, RT_ALL_FIBS); - if (dstaddr != NULL) { - if (dstaddr->ifa_ifp == ifp) { - ifa_free(dstaddr); - return (1); + if (ifp->if_flags & IFF_POINTOPOINT) { + IF_ADDR_RLOCK(ifp); + TAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link) { + if (ifa->ifa_addr->sa_family != addr->sin6_family) + continue; + if (ifa->ifa_dstaddr != NULL && + sa_equal(addr, ifa->ifa_dstaddr)) { + IF_ADDR_RUNLOCK(ifp); + return 1; + } } - ifa_free(dstaddr); + IF_ADDR_RUNLOCK(ifp); } /* @@ -1485,7 +1501,7 @@ nd6_free(struct llentry **lnp, int gc) /* * We need to unlock to avoid a LOR with rt6_flush() with the * rnh and for the calls to pfxlist_onlink_check() and - * defrouter_select() in the block further down for calls + * defrouter_select_fib() in the block further down for calls * into nd6_lookup(). We still hold a ref. */ LLE_WUNLOCK(ln); @@ -1500,7 +1516,7 @@ nd6_free(struct llentry **lnp, int gc) if (dr) { /* - * Since defrouter_select() does not affect the + * Since defrouter_select_fib() does not affect the * on-link determination and MIP6 needs the check * before the default router selection, we perform * the check now. @@ -1510,7 +1526,7 @@ nd6_free(struct llentry **lnp, int gc) /* * Refresh default router list. */ - defrouter_select(); + defrouter_select_fib(dr->ifp->if_fib); } /* @@ -2104,11 +2120,11 @@ nd6_cache_lladdr(struct ifnet *ifp, stru * Question: can we restrict the first condition to the "is_newentry" * case? * XXX: when we hear an RA from a new router with the link-layer - * address option, defrouter_select() is called twice, since + * address option, defrouter_select_fib() is called twice, since * defrtrlist_update called the function as well. However, I believe * we can compromise the overhead, since it only happens the first * time. - * XXX: although defrouter_select() should not have a bad effect + * XXX: although defrouter_select_fib() should not have a bad effect * for those are not autoconfigured hosts, we explicitly avoid such * cases for safety. */ @@ -2117,7 +2133,7 @@ nd6_cache_lladdr(struct ifnet *ifp, stru /* * guaranteed recursion */ - defrouter_select(); + defrouter_select_fib(ifp->if_fib); } } Modified: stable/11/sys/netinet6/nd6.h ============================================================================== --- stable/11/sys/netinet6/nd6.h Mon Apr 17 19:10:12 2017 (r317066) +++ stable/11/sys/netinet6/nd6.h Mon Apr 17 20:13:20 2017 (r317067) @@ -469,6 +469,7 @@ void nd6_dad_stop(struct ifaddr *); void nd6_rs_input(struct mbuf *, int, int); void nd6_ra_input(struct mbuf *, int, int); void defrouter_reset(void); +void defrouter_select_fib(int fibnum); void defrouter_select(void); void defrouter_ref(struct nd_defrouter *); void defrouter_rele(struct nd_defrouter *); Modified: stable/11/sys/netinet6/nd6_nbr.c ============================================================================== --- stable/11/sys/netinet6/nd6_nbr.c Mon Apr 17 19:10:12 2017 (r317066) +++ stable/11/sys/netinet6/nd6_nbr.c Mon Apr 17 20:13:20 2017 (r317067) @@ -262,8 +262,7 @@ nd6_ns_input(struct mbuf *m, int off, in bzero(&info, sizeof(info)); info.rti_info[RTAX_GATEWAY] = (struct sockaddr *)&rt_gateway; - /* Always use the default FIB. */ - if (rib_lookup_info(RT_DEFAULT_FIB, (struct sockaddr *)&dst6, + if (rib_lookup_info(ifp->if_fib, (struct sockaddr *)&dst6, 0, 0, &info) == 0) { if ((info.rti_flags & RTF_ANNOUNCE) != 0 && rt_gateway.sdl_family == AF_LINK) { @@ -485,7 +484,7 @@ nd6_ns_output_fib(struct ifnet *ifp, con uint32_t scopeid; in6_splitscope(&ip6->ip6_dst, &dst6, &scopeid); - error = in6_selectsrc_addr(RT_DEFAULT_FIB, &dst6, + error = in6_selectsrc_addr(fibnum, &dst6, scopeid, ifp, &src6, NULL); if (error) { char ip6buf[INET6_ADDRSTRLEN]; @@ -982,7 +981,7 @@ nd6_na_output_fib(struct ifnet *ifp, con * Select a source whose scope is the same as that of the dest. */ in6_splitscope(&daddr6, &dst6, &scopeid); - error = in6_selectsrc_addr(RT_DEFAULT_FIB, &dst6, + error = in6_selectsrc_addr(fibnum, &dst6, scopeid, ifp, &src6, NULL); if (error) { char ip6buf[INET6_ADDRSTRLEN]; Modified: stable/11/sys/netinet6/nd6_rtr.c ============================================================================== --- stable/11/sys/netinet6/nd6_rtr.c Mon Apr 17 19:10:12 2017 (r317066) +++ stable/11/sys/netinet6/nd6_rtr.c Mon Apr 17 20:13:20 2017 (r317067) @@ -500,7 +500,7 @@ defrouter_addreq(struct nd_defrouter *ne error = in6_rtrequest(RTM_ADD, (struct sockaddr *)&def, (struct sockaddr *)&gate, (struct sockaddr *)&mask, - RTF_GATEWAY, &newrt, RT_DEFAULT_FIB); + RTF_GATEWAY, &newrt, new->ifp->if_fib); if (newrt) { nd6_rtmsg(RTM_ADD, newrt); /* tell user process */ RTFREE(newrt); @@ -551,8 +551,8 @@ defrouter_rele(struct nd_defrouter *dr) /* * Remove the default route for a given router. - * This is just a subroutine function for defrouter_select(), and should - * not be called from anywhere else. + * This is just a subroutine function for defrouter_select_fib(), and + * should not be called from anywhere else. */ static void defrouter_delreq(struct nd_defrouter *dr) @@ -571,7 +571,7 @@ defrouter_delreq(struct nd_defrouter *dr in6_rtrequest(RTM_DELETE, (struct sockaddr *)&def, (struct sockaddr *)&gate, - (struct sockaddr *)&mask, RTF_GATEWAY, &oldrt, RT_DEFAULT_FIB); + (struct sockaddr *)&mask, RTF_GATEWAY, &oldrt, dr->ifp->if_fib); if (oldrt) { nd6_rtmsg(RTM_DELETE, oldrt); RTFREE(oldrt); @@ -698,11 +698,11 @@ defrouter_del(struct nd_defrouter *dr) /* * If the router is the primary one, choose a new one. - * Note that defrouter_select() will remove the current gateway - * from the routing table. + * Note that defrouter_select_fib() will remove the current + * gateway from the routing table. */ if (deldr) - defrouter_select(); + defrouter_select_fib(deldr->ifp->if_fib); /* * Release the list reference. @@ -730,13 +730,23 @@ defrouter_del(struct nd_defrouter *dr) * even when the multipath routing is available, because we're not sure about * the benefits for stub hosts comparing to the risk of making the code * complicated and the possibility of introducing bugs. + * + * We maintain a single list of routers for multiple FIBs, only considering one + * at a time based on the receiving interface's FIB. If @fibnum is RT_ALL_FIBS, + * we do the whole thing multiple times. */ void -defrouter_select(void) +defrouter_select_fib(int fibnum) { struct nd_defrouter *dr, *selected_dr, *installed_dr; struct llentry *ln = NULL; + if (fibnum == RT_ALL_FIBS) { + for (fibnum = 0; fibnum < rt_numfibs; fibnum++) { + defrouter_select_fib(fibnum); + } + } + ND6_RLOCK(); /* * Let's handle easy case (3) first: @@ -755,7 +765,7 @@ defrouter_select(void) selected_dr = installed_dr = NULL; TAILQ_FOREACH(dr, &V_nd_defrouter, dr_entry) { IF_AFDATA_RLOCK(dr->ifp); - if (selected_dr == NULL && + if (selected_dr == NULL && dr->ifp->if_fib == fibnum && (ln = nd6_lookup(&dr->rtaddr, 0, dr->ifp)) && ND6_IS_LLINFO_PROBREACH(ln)) { selected_dr = dr; @@ -767,14 +777,17 @@ defrouter_select(void) ln = NULL; } - if (dr->installed) { + if (dr->installed && dr->ifp->if_fib == fibnum) { if (installed_dr == NULL) { installed_dr = dr; defrouter_ref(installed_dr); } else { - /* this should not happen. warn for diagnosis. */ - log(LOG_ERR, - "defrouter_select: more than one router is installed\n"); + /* + * this should not happen. + * warn for diagnosis. + */ + log(LOG_ERR, "defrouter_select_fib: more than " + "one router is installed\n"); } } } @@ -789,14 +802,24 @@ defrouter_select(void) if (selected_dr == NULL) { if (installed_dr == NULL || TAILQ_NEXT(installed_dr, dr_entry) == NULL) - selected_dr = TAILQ_FIRST(&V_nd_defrouter); + dr = TAILQ_FIRST(&V_nd_defrouter); else - selected_dr = TAILQ_NEXT(installed_dr, dr_entry); - defrouter_ref(selected_dr); + dr = TAILQ_NEXT(installed_dr, dr_entry); + + /* Ensure we select a router for this FIB. */ + TAILQ_FOREACH_FROM(dr, &V_nd_defrouter, dr_entry) { + if (dr->ifp->if_fib == fibnum) { + selected_dr = dr; + defrouter_ref(selected_dr); + break; + } + } } else if (installed_dr != NULL) { IF_AFDATA_RLOCK(installed_dr->ifp); - if ((ln = nd6_lookup(&installed_dr->rtaddr, 0, installed_dr->ifp)) && + if ((ln = nd6_lookup(&installed_dr->rtaddr, 0, + installed_dr->ifp)) && ND6_IS_LLINFO_PROBREACH(ln) && + installed_dr->ifp->if_fib == fibnum && rtpref(selected_dr) <= rtpref(installed_dr)) { defrouter_rele(selected_dr); selected_dr = installed_dr; @@ -808,18 +831,30 @@ defrouter_select(void) ND6_RUNLOCK(); /* - * If the selected router is different than the installed one, - * remove the installed router and install the selected one. - * Note that the selected router is never NULL here. + * If we selected a router for this FIB and it's different + * than the installed one, remove the installed router and + * install the selected one in its place. */ if (installed_dr != selected_dr) { if (installed_dr != NULL) { defrouter_delreq(installed_dr); defrouter_rele(installed_dr); } - defrouter_addreq(selected_dr); + if (selected_dr != NULL) + defrouter_addreq(selected_dr); } - defrouter_rele(selected_dr); + if (selected_dr != NULL) + defrouter_rele(selected_dr); +} + +/* + * Maintain old KPI for default router selection. + * If unspecified, we can re-select routers for all FIBs. + */ +void +defrouter_select(void) +{ + defrouter_select_fib(RT_ALL_FIBS); } /* @@ -942,7 +977,7 @@ restart: V_nd6_list_genid++; ND6_WUNLOCK(); - defrouter_select(); + defrouter_select_fib(new->ifp->if_fib); return (n); } @@ -1731,7 +1766,7 @@ nd6_prefix_onlink_rtrequest(struct nd_pr struct rtentry *rt; struct sockaddr_in6 mask6; u_long rtflags; - int error, a_failure, fibnum; + int error, a_failure, fibnum, maxfib; /* * in6_ifinit() sets nd6_rtrequest to ifa_rtrequest for all ifaddrs. @@ -1742,8 +1777,15 @@ nd6_prefix_onlink_rtrequest(struct nd_pr mask6.sin6_addr = pr->ndpr_mask; rtflags = (ifa->ifa_flags & ~IFA_RTSELF) | RTF_UP; + if(V_rt_add_addr_allfibs) { + fibnum = 0; + maxfib = rt_numfibs; + } else { + fibnum = ifa->ifa_ifp->if_fib; + maxfib = fibnum + 1; + } a_failure = 0; - for (fibnum = 0; fibnum < rt_numfibs; fibnum++) { + for (; fibnum < maxfib; fibnum++) { rt = NULL; error = in6_rtrequest(RTM_ADD, @@ -1831,6 +1873,10 @@ nd6_prefix_onlink(struct nd_prefix *pr) if ((opr->ndpr_stateflags & NDPRF_ONLINK) == 0) continue; + if (!V_rt_add_addr_allfibs && + opr->ndpr_ifp->if_fib != pr->ndpr_ifp->if_fib) + continue; + if (opr->ndpr_plen == pr->ndpr_plen && in6_are_prefix_equal(&pr->ndpr_prefix.sin6_addr, &opr->ndpr_prefix.sin6_addr, pr->ndpr_plen)) { @@ -1891,7 +1937,7 @@ nd6_prefix_offlink(struct nd_prefix *pr) struct rtentry *rt; char ip6buf[INET6_ADDRSTRLEN]; uint64_t genid; - int fibnum, a_failure; + int fibnum, maxfib, a_failure; ND6_ONLINK_LOCK_ASSERT(); ND6_UNLOCK_ASSERT(); @@ -1909,8 +1955,16 @@ nd6_prefix_offlink(struct nd_prefix *pr) mask6.sin6_len = sizeof(sa6); bcopy(&pr->ndpr_mask, &mask6.sin6_addr, sizeof(struct in6_addr)); + if (V_rt_add_addr_allfibs) { + fibnum = 0; + maxfib = rt_numfibs; + } else { + fibnum = ifp->if_fib; + maxfib = fibnum + 1; + } + a_failure = 0; - for (fibnum = 0; fibnum < rt_numfibs; fibnum++) { + for (; fibnum < maxfib; fibnum++) { rt = NULL; error = in6_rtrequest(RTM_DELETE, (struct sockaddr *)&sa6, NULL, (struct sockaddr *)&mask6, 0, &rt, fibnum); Modified: stable/11/tests/sys/netinet/fibs_test.sh ============================================================================== --- stable/11/tests/sys/netinet/fibs_test.sh Mon Apr 17 19:10:12 2017 (r317066) +++ stable/11/tests/sys/netinet/fibs_test.sh Mon Apr 17 20:13:20 2017 (r317067) @@ -39,8 +39,7 @@ # arpresolve only checked the default route. # # Outline: -# Create two tap(4) interfaces -# Simulate a crossover cable between them by using net/socat +# Create two connected epair(4) interfaces # Use nping (from security/nmap) to send an ICMP echo request from one # interface to the other, spoofing the source IP. The source IP must be # spoofed, or else it will already have an entry in the arp table. @@ -51,7 +50,7 @@ arpresolve_checks_interface_fib_head() atf_set "descr" "arpresolve should check the interface fib, not the default fib, for routes" atf_set "require.user" "root" atf_set "require.config" "fibs" - atf_set "require.progs" "socat nping" + atf_set "require.progs" "nping" } arpresolve_checks_interface_fib_body() { @@ -74,19 +73,13 @@ arpresolve_checks_interface_fib_body() fi get_fibs 2 - # Configure TAP interfaces - setup_tap "$FIB0" ${ADDR0} ${MASK0} - TAP0=$TAP - setup_tap "$FIB1" ${ADDR1} ${MASK1} - TAP1=$TAP - - # Simulate a crossover cable - socat /dev/${TAP0} /dev/${TAP1} & - SOCAT_PID=$! - echo ${SOCAT_PID} >> "processes_to_kill" + # Configure epair interfaces + get_epair + setup_iface "$EPAIRA" "$FIB0" inet ${ADDR0} ${MASK0} + setup_iface "$EPAIRB" "$FIB1" inet ${ADDR1} ${MASK1} # Send an ICMP echo request with a spoofed source IP - setfib 2 nping -c 1 -e ${TAP0} -S ${SPOOF_ADDR} \ + setfib "$FIB0" nping -c 1 -e ${EPAIRA} -S ${SPOOF_ADDR} \ --source-mac ${SPOOF_MAC} --icmp --icmp-type "echo-request" \ --icmp-code 0 --icmp-id 0xdead --icmp-seq 1 --data 0xbeef \ ${ADDR1} @@ -94,17 +87,11 @@ arpresolve_checks_interface_fib_body() # characteristic error message dmesg | grep "llinfo.*${SPOOF_ADDR}" # Check that the ARP entry exists - atf_check -o match:"${SPOOF_ADDR}.*expires" setfib 3 arp ${SPOOF_ADDR} + atf_check -o match:"${SPOOF_ADDR}.*expires" setfib "$FIB1" arp ${SPOOF_ADDR} } arpresolve_checks_interface_fib_cleanup() { - if [ -f processes_to_kill ]; then - for pid in $(cat processes_to_kill); do - kill "${pid}" - done - rm -f processes_to_kill - fi - cleanup_tap + cleanup_ifaces } @@ -112,7 +99,7 @@ arpresolve_checks_interface_fib_cleanup( atf_test_case loopback_and_network_routes_on_nondefault_fib cleanup loopback_and_network_routes_on_nondefault_fib_head() { - atf_set "descr" "When creating and deleting loopback routes, use the interface's fib" + atf_set "descr" "When creating and deleting loopback IPv4 routes, use the interface's fib" atf_set "require.user" "root" atf_set "require.config" "fibs" } @@ -132,7 +119,7 @@ loopback_and_network_routes_on_nondefaul get_fibs 1 # Configure a TAP interface - setup_tap ${FIB0} ${ADDR} ${MASK} + setup_tap ${FIB0} inet ${ADDR} ${MASK} # Check whether the host route exists in only the correct FIB setfib ${FIB0} netstat -rn -f inet | grep -q "^${ADDR}.*UHS.*lo0" @@ -156,14 +143,71 @@ loopback_and_network_routes_on_nondefaul setfib 0 netstat -rn -f inet | \ grep -q "^${SUBNET}/${MASK}.*${TAPD}" if [ 0 -eq $? ]; then - setfib ${FIB0} netstat -rn -f inet + setfib 0 netstat -rn -f inet atf_fail "Network route appeared in the wrong FIB" fi } loopback_and_network_routes_on_nondefault_fib_cleanup() { - cleanup_tap + cleanup_ifaces +} + +atf_test_case loopback_and_network_routes_on_nondefault_fib_inet6 cleanup +loopback_and_network_routes_on_nondefault_fib_inet6_head() +{ + atf_set "descr" "When creating and deleting loopback IPv6 routes, use the interface's fib" + atf_set "require.user" "root" + atf_set "require.config" "fibs" +} + +loopback_and_network_routes_on_nondefault_fib_inet6_body() +{ + # Configure the TAP interface to use a nonrouteable RFC3849 + # address and a non-default fib + ADDR="2001:db8::2" + SUBNET="2001:db8::" + MASK="64" + + # Check system configuration + if [ 0 != `sysctl -n net.add_addr_allfibs` ]; then + atf_skip "This test requires net.add_addr_allfibs=0" + fi + get_fibs 1 + + # Configure a TAP interface + setup_tap ${FIB0} inet6 ${ADDR} ${MASK} + + # Check whether the host route exists in only the correct FIB + setfib ${FIB0} netstat -rn -f inet6 | grep -q "^${ADDR}.*UHS.*lo0" + if [ 0 -ne $? ]; then + setfib ${FIB0} netstat -rn -f inet6 + atf_fail "Host route did not appear in the correct FIB" + fi + setfib 0 netstat -rn -f inet6 | grep -q "^${ADDR}.*UHS.*lo0" + if [ 0 -eq $? ]; then + setfib 0 netstat -rn -f inet6 + atf_fail "Host route appeared in the wrong FIB" + fi + + # Check whether the network route exists in only the correct FIB + setfib ${FIB0} netstat -rn -f inet6 | \ + grep -q "^${SUBNET}/${MASK}.*${TAPD}" + if [ 0 -ne $? ]; then + setfib ${FIB0} netstat -rn -f inet6 + atf_fail "Network route did not appear in the correct FIB" + fi + setfib 0 netstat -rn -f inet6 | \ + grep -q "^${SUBNET}/${MASK}.*${TAPD}" + if [ 0 -eq $? ]; then + setfib 0 netstat -rn -f inet6 + atf_fail "Network route appeared in the wrong FIB" + fi +} + +loopback_and_network_routes_on_nondefault_fib_inet6_cleanup() +{ + cleanup_ifaces } @@ -171,7 +215,7 @@ loopback_and_network_routes_on_nondefaul atf_test_case default_route_with_multiple_fibs_on_same_subnet cleanup default_route_with_multiple_fibs_on_same_subnet_head() { - atf_set "descr" "Multiple interfaces on the same subnet but with different fibs can both have default routes" + atf_set "descr" "Multiple interfaces on the same subnet but with different fibs can both have default IPv4 routes" atf_set "require.user" "root" atf_set "require.config" "fibs" } @@ -193,9 +237,9 @@ default_route_with_multiple_fibs_on_same get_fibs 2 # Configure TAP interfaces - setup_tap "$FIB0" ${ADDR0} ${MASK} + setup_tap "$FIB0" inet ${ADDR0} ${MASK} TAP0=$TAP - setup_tap "$FIB1" ${ADDR1} ${MASK} + setup_tap "$FIB1" inet ${ADDR1} ${MASK} TAP1=$TAP # Attempt to add default routes @@ -212,7 +256,54 @@ default_route_with_multiple_fibs_on_same default_route_with_multiple_fibs_on_same_subnet_cleanup() { - cleanup_tap + cleanup_ifaces +} + +atf_test_case default_route_with_multiple_fibs_on_same_subnet_inet6 cleanup +default_route_with_multiple_fibs_on_same_subnet_inet6_head() +{ + atf_set "descr" "Multiple interfaces on the same subnet but with different fibs can both have default IPv6 routes" + atf_set "require.user" "root" + atf_set "require.config" "fibs" +} + +default_route_with_multiple_fibs_on_same_subnet_inet6_body() +{ + # Configure the TAP interfaces to use nonrouteable RFC3849 + # addresses and non-default FIBs + ADDR0="2001:db8::2" + ADDR1="2001:db8::3" + GATEWAY="2001:db8::1" + SUBNET="2001:db8::" + MASK="64" + + # Check system configuration + if [ 0 != `sysctl -n net.add_addr_allfibs` ]; then + atf_skip "This test requires net.add_addr_allfibs=0" + fi + get_fibs 2 + + # Configure TAP interfaces + setup_tap "$FIB0" inet6 ${ADDR0} ${MASK} + TAP0=$TAP + setup_tap "$FIB1" inet6 ${ADDR1} ${MASK} + TAP1=$TAP + + # Attempt to add default routes + setfib ${FIB0} route -6 add default ${GATEWAY} + setfib ${FIB1} route -6 add default ${GATEWAY} + + # Verify that the default route exists for both fibs, with their + # respective interfaces. + atf_check -o match:"^default.*${TAP0}$" \ + setfib ${FIB0} netstat -rn -f inet6 + atf_check -o match:"^default.*${TAP1}$" \ + setfib ${FIB1} netstat -rn -f inet6 +} + +default_route_with_multiple_fibs_on_same_subnet_inet6_cleanup() +{ + cleanup_ifaces } @@ -223,7 +314,7 @@ default_route_with_multiple_fibs_on_same atf_test_case same_ip_multiple_ifaces_fib0 cleanup same_ip_multiple_ifaces_fib0_head() { - atf_set "descr" "Can remove an IP alias from an interface when the same IP is also assigned to another interface." + atf_set "descr" "Can remove an IPv4 alias from an interface when the same IPv4 is also assigned to another interface." atf_set "require.user" "root" atf_set "require.config" "fibs" } @@ -237,22 +328,22 @@ same_ip_multiple_ifaces_fib0_body() # of net.add_addr_allfibs # Setup the interfaces, then remove one alias. It should not panic. - setup_tap 0 ${ADDR} ${MASK0} + setup_tap 0 inet ${ADDR} ${MASK0} TAP0=${TAP} - setup_tap 0 ${ADDR} ${MASK1} + setup_tap 0 inet ${ADDR} ${MASK1} TAP1=${TAP} ifconfig ${TAP1} -alias ${ADDR} # Do it again, in the opposite order. It should not panic. - setup_tap 0 ${ADDR} ${MASK0} + setup_tap 0 inet ${ADDR} ${MASK0} TAP0=${TAP} - setup_tap 0 ${ADDR} ${MASK1} + setup_tap 0 inet ${ADDR} ${MASK1} TAP1=${TAP} ifconfig ${TAP0} -alias ${ADDR} } same_ip_multiple_ifaces_fib0_cleanup() { - cleanup_tap + cleanup_ifaces } # Regression test for PR kern/189088 @@ -266,7 +357,7 @@ same_ip_multiple_ifaces_fib0_cleanup() atf_test_case same_ip_multiple_ifaces cleanup same_ip_multiple_ifaces_head() { - atf_set "descr" "Can remove an IP alias from an interface when the same IP is also assigned to another interface, on non-default FIBs." + atf_set "descr" "Can remove an IPv4 alias from an interface when the same address is also assigned to another interface, on non-default FIBs." atf_set "require.user" "root" atf_set "require.config" "fibs" } @@ -282,18 +373,18 @@ same_ip_multiple_ifaces_body() get_fibs 2 # Setup the interfaces, then remove one alias. It should not panic. - setup_tap ${FIB0} ${ADDR} ${MASK0} + setup_tap ${FIB0} inet ${ADDR} ${MASK0} TAP0=${TAP} - setup_tap ${FIB1} ${ADDR} ${MASK1} + setup_tap ${FIB1} inet ${ADDR} ${MASK1} TAP1=${TAP} ifconfig ${TAP1} -alias ${ADDR} atf_check -o not-match:"^${ADDR}[[:space:]]" \ setfib ${FIB1} netstat -rn -f inet # Do it again, in the opposite order. It should not panic. - setup_tap ${FIB0} ${ADDR} ${MASK0} + setup_tap ${FIB0} inet ${ADDR} ${MASK0} TAP0=${TAP} - setup_tap ${FIB1} ${ADDR} ${MASK1} + setup_tap ${FIB1} inet ${ADDR} ${MASK1} TAP1=${TAP} ifconfig ${TAP0} -alias ${ADDR} atf_check -o not-match:"^${ADDR}[[:space:]]" \ @@ -303,16 +394,145 @@ same_ip_multiple_ifaces_cleanup() { # Due to PR kern/189088, we must destroy the interfaces in LIFO order # in order for the routes to be correctly cleaned up. - for TAPD in `tail -r "tap_devices_to_cleanup"`; do + for TAPD in `tail -r "ifaces_to_cleanup"`; do + echo ifconfig ${TAPD} destroy ifconfig ${TAPD} destroy done } +atf_test_case same_ip_multiple_ifaces_inet6 cleanup +same_ip_multiple_ifaces_inet6_head() +{ + atf_set "descr" "Can remove an IPv6 alias from an interface when the same address is also assigned to another interface, on non-default FIBs." + atf_set "require.user" "root" + atf_set "require.config" "fibs" +} +same_ip_multiple_ifaces_inet6_body() +{ + ADDR="2001:db8::2" + MASK0="64" + MASK1="128" + + # Unlike most of the tests in this file, this is applicable regardless + # of net.add_addr_allfibs + get_fibs 2 + + # Setup the interfaces, then remove one alias. It should not panic. + setup_tap ${FIB0} inet6 ${ADDR} ${MASK0} + TAP0=${TAP} + setup_tap ${FIB1} inet6 ${ADDR} ${MASK1} + TAP1=${TAP} + atf_check -s exit:0 ifconfig ${TAP1} inet6 ${ADDR} -alias + atf_check -o not-match:"^${ADDR}[[:space:]]" \ + setfib ${FIB1} netstat -rn -f inet6 + ifconfig ${TAP1} destroy + ifconfig ${TAP0} destroy + + # Do it again, in the opposite order. It should not panic. + setup_tap ${FIB0} inet6 ${ADDR} ${MASK0} + TAP0=${TAP} + setup_tap ${FIB1} inet6 ${ADDR} ${MASK1} + TAP1=${TAP} + atf_check -s exit:0 ifconfig ${TAP0} inet6 ${ADDR} -alias + atf_check -o not-match:"^${ADDR}[[:space:]]" \ + setfib ${FIB0} netstat -rn -f inet6 +} +same_ip_multiple_ifaces_inet6_cleanup() +{ + cleanup_ifaces +} + +atf_test_case slaac_on_nondefault_fib6 cleanup +slaac_on_nondefault_fib6_head() +{ + atf_set "descr" "SLAAC correctly installs routes on non-default FIBs" + atf_set "require.user" "root" + atf_set "require.config" "fibs" "allow_sysctl_side_effects" +} +slaac_on_nondefault_fib6_body() +{ + # Configure the epair interfaces to use nonrouteable RFC3849 + # addresses and non-default FIBs + PREFIX="2001:db8:$(printf "%x" `jot -r 1 0 65535`):$(printf "%x" `jot -r 1 0 65535`)" + ADDR="$PREFIX::2" + GATEWAY="$PREFIX::1" + SUBNET="$PREFIX:" + MASK="64" + + # Check system configuration + if [ 0 != `sysctl -n net.add_addr_allfibs` ]; then + atf_skip "This test requires net.add_addr_allfibs=0" + fi + get_fibs 2 + + sysctl -n "net.inet6.ip6.rfc6204w3" >> "rfc6204w3.state" + sysctl -n "net.inet6.ip6.forwarding" >> "forwarding.state" + # Enable forwarding so the kernel will send RAs + sysctl net.inet6.ip6.forwarding=1 + # Enable RFC6204W3 mode so the kernel will enable default router + # selection while also forwarding packets + sysctl net.inet6.ip6.rfc6204w3=1 + + # Configure epair interfaces + get_epair + setup_iface "$EPAIRA" "$FIB0" inet6 ${ADDR} ${MASK} + echo setfib $FIB1 ifconfig "$EPAIRB" inet6 -ifdisabled accept_rtadv fib $FIB1 up + setfib $FIB1 ifconfig "$EPAIRB" inet6 -ifdisabled accept_rtadv fib $FIB1 up + rtadvd -p rtadvd.pid -C rtadvd.sock -c /dev/null "$EPAIRA" + rtsol "$EPAIRB" + + # Check SLAAC address + atf_check -o match:"inet6 ${SUBNET}.*prefixlen ${MASK}.*autoconf" \ + ifconfig "$EPAIRB" + # Check local route + atf_check -o match:"${SUBNET}.*\.*lo0" \ + netstat -rnf inet6 -F $FIB1 + # Check subnet route + atf_check -o match:"${SUBNET}:/${MASK}.*\.*$EPAIRB" \ + netstat -rnf inet6 -F $FIB1 + # Check default route + atf_check -o match:"default.*\.*$EPAIRB" \ + netstat -rnf inet6 -F $FIB1 + + # Check that none of the above routes appeared on other routes + for fib in $( seq 0 $(($(sysctl -n net.fibs) - 1))); do + if [ "$fib" = "$FIB1" -o "$fib" = "$FIB0" ]; then + continue + fi + atf_check -o not-match:"${SUBNET}.*\.*lo0" \ + netstat -rnf inet6 -F $fib + atf_check -o not-match:"${SUBNET}:/${MASK}.*\.*$EPAIRB" \ + netstat -rnf inet6 -F $fib + atf_check -o not-match:"default.*\.*$EPAIRB" \ + netstat -rnf inet6 -F $fib + done +} +slaac_on_nondefault_fib6_cleanup() +{ + if [ -f "rtadvd.pid" ]; then + # rtadvd can take a long time to shutdown. Use SIGKILL to kill + # it right away. The downside to using SIGKILL is that it + # won't send final RAs to all interfaces, but we don't care + # because we're about to destroy its interface anyway. + pkill -kill -F rtadvd.pid + rm -f rtadvd.pid + fi + cleanup_ifaces + if [ -f "forwarding.state" ] ; then + sysctl "net.inet6.ip6.forwarding"=`cat "forwarding.state"` + rm "forwarding.state" + fi + if [ -f "rfc6204w3.state" ] ; then + sysctl "net.inet6.ip6.rfc6204w3"=`cat "rfc6204w3.state"` + rm "rfc6204w3.state" + fi +} + # Regression test for kern/187550 atf_test_case subnet_route_with_multiple_fibs_on_same_subnet cleanup subnet_route_with_multiple_fibs_on_same_subnet_head() { - atf_set "descr" "Multiple FIBs can have subnet routes for the same subnet" + atf_set "descr" "Multiple FIBs can have IPv4 subnet routes for the same subnet" atf_set "require.user" "root" atf_set "require.config" "fibs" } @@ -333,8 +553,8 @@ subnet_route_with_multiple_fibs_on_same_ get_fibs 2 # Configure TAP interfaces - setup_tap "$FIB0" ${ADDR0} ${MASK} - setup_tap "$FIB1" ${ADDR1} ${MASK} + setup_tap "$FIB0" inet ${ADDR0} ${MASK} + setup_tap "$FIB1" inet ${ADDR1} ${MASK} # Check that a subnet route exists on both fibs atf_check -o ignore setfib "$FIB0" route get $ADDR1 @@ -343,7 +563,44 @@ subnet_route_with_multiple_fibs_on_same_ subnet_route_with_multiple_fibs_on_same_subnet_cleanup() { - cleanup_tap + cleanup_ifaces +} + +atf_test_case subnet_route_with_multiple_fibs_on_same_subnet_inet6 cleanup +subnet_route_with_multiple_fibs_on_same_subnet_inet6_head() *** DIFF OUTPUT TRUNCATED AT 1000 LINES ***