From owner-dev-commits-src-all@freebsd.org Wed Mar 10 21:28:27 2021 Return-Path: Delivered-To: dev-commits-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0E6055782A7; Wed, 10 Mar 2021 21:28:27 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DwlYk747hz4hdf; Wed, 10 Mar 2021 21:28:26 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E5A9318905; Wed, 10 Mar 2021 21:28:26 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 12ALSQWY053341; Wed, 10 Mar 2021 21:28:26 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 12ALSQjR053340; Wed, 10 Mar 2021 21:28:26 GMT (envelope-from git) Date: Wed, 10 Mar 2021 21:28:26 GMT Message-Id: <202103102128.12ALSQjR053340@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: "Alexander V. Chernikov" Subject: git: b1d63265ac39 - main - Flush remaining routes from the routing table during VNET shutdown. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: melifaro X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: b1d63265ac399112b3bca36c3d75df1a3c2c8102 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-all@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commit messages for all branches of the src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Mar 2021 21:28:27 -0000 The branch main has been updated by melifaro: URL: https://cgit.FreeBSD.org/src/commit/?id=b1d63265ac399112b3bca36c3d75df1a3c2c8102 commit b1d63265ac399112b3bca36c3d75df1a3c2c8102 Author: Alexander V. Chernikov AuthorDate: 2021-03-08 21:35:41 +0000 Commit: Alexander V. Chernikov CommitDate: 2021-03-10 21:10:14 +0000 Flush remaining routes from the routing table during VNET shutdown. Summary: This fixes rtentry leak for the cloned interfaces created inside the VNET. PR: 253998 Reported by: rashey at superbox.pl MFC after: 3 days Loopback teardown order is `SI_SUB_INIT_IF`, which happens after `SI_SUB_PROTO_DOMAIN` (route table teardown). Thus, any route table operations are too late to schedule. As the intent of the vnet teardown procedures to minimise the amount of effort by doing global cleanups instead of per-interface ones, address this by adding a relatively light-weight routing table cleanup function, `rib_flush_routes()`. It removes all remaining routes from the routing table and schedules the deletion, which will happen later, when `rtables_destroy()` waits for the current epoch to finish. Test Plan: ``` set_skip:set_skip_group_lo -> passed [0.053s] tail -n 200 /var/log/messages | grep rtentry ``` Reviewers: #network, kp, bz Reviewed By: kp Subscribers: imp, ae Differential Revision: https://reviews.freebsd.org/D29116 --- sys/net/route.c | 15 --------------- sys/net/route.h | 2 +- sys/net/route/route_ctl.c | 36 ++++++++++++++++++++++++++++++++++++ sys/netinet/ip_input.c | 6 +----- sys/netinet6/ip6_input.c | 5 +++-- 5 files changed, 41 insertions(+), 23 deletions(-) diff --git a/sys/net/route.c b/sys/net/route.c index d2b405fafcbf..2416aa9a983f 100644 --- a/sys/net/route.c +++ b/sys/net/route.c @@ -493,21 +493,6 @@ rt_ifdelroute(const struct rtentry *rt, const struct nhop_object *nh, void *arg) return (1); } -/* - * Delete all remaining routes using this interface - * Unfortuneatly the only way to do this is to slog through - * the entire routing table looking for routes which point - * to this interface...oh well... - */ -void -rt_flushifroutes_af(struct ifnet *ifp, int af) -{ - KASSERT((af >= 1 && af <= AF_MAX), ("%s: af %d not >= 1 and <= %d", - __func__, af, AF_MAX)); - - rib_foreach_table_walk_del(af, rt_ifdelroute, ifp); -} - void rt_flushifroutes(struct ifnet *ifp) { diff --git a/sys/net/route.h b/sys/net/route.h index ab6e1aabc5ae..3fdca303596e 100644 --- a/sys/net/route.h +++ b/sys/net/route.h @@ -429,7 +429,6 @@ struct sockaddr *rtsock_fix_netmask(const struct sockaddr *dst, void rt_updatemtu(struct ifnet *); -void rt_flushifroutes_af(struct ifnet *, int); void rt_flushifroutes(struct ifnet *ifp); /* XXX MRT NEW VERSIONS THAT USE FIBs @@ -442,6 +441,7 @@ int rib_lookup_info(uint32_t, const struct sockaddr *, uint32_t, uint32_t, void rib_free_info(struct rt_addrinfo *info); /* New API */ +void rib_flush_routes_family(int family); struct nhop_object *rib_lookup(uint32_t fibnum, const struct sockaddr *dst, uint32_t flags, uint32_t flowid); #endif diff --git a/sys/net/route/route_ctl.c b/sys/net/route/route_ctl.c index e07a94295b89..d9fd1aa18c92 100644 --- a/sys/net/route/route_ctl.c +++ b/sys/net/route/route_ctl.c @@ -1328,6 +1328,42 @@ rib_walk_del(u_int fibnum, int family, rib_filter_f_t *filter_f, void *arg, bool NET_EPOCH_EXIT(et); } +static int +rt_delete_unconditional(struct radix_node *rn, void *arg) +{ + struct rtentry *rt = RNTORT(rn); + struct rib_head *rnh = (struct rib_head *)arg; + + rn = rnh->rnh_deladdr(rt_key(rt), rt_mask(rt), &rnh->head); + if (RNTORT(rn) == rt) + rtfree(rt); + + return (0); +} + +/* + * Removes all routes from the routing table without executing notifications. + * rtentres will be removed after the end of a current epoch. + */ +static void +rib_flush_routes(struct rib_head *rnh) +{ + RIB_WLOCK(rnh); + rnh->rnh_walktree(&rnh->head, rt_delete_unconditional, rnh); + RIB_WUNLOCK(rnh); +} + +void +rib_flush_routes_family(int family) +{ + struct rib_head *rnh; + + for (uint32_t fibnum = 0; fibnum < rt_numfibs; fibnum++) { + if ((rnh = rt_tables_get_rnh(fibnum, family)) != NULL) + rib_flush_routes(rnh); + } +} + static void rib_notify(struct rib_head *rnh, enum rib_subscription_type type, struct rib_cmd_info *rc) diff --git a/sys/netinet/ip_input.c b/sys/netinet/ip_input.c index be21decff6cb..a85f8ac7b567 100644 --- a/sys/netinet/ip_input.c +++ b/sys/netinet/ip_input.c @@ -379,7 +379,6 @@ ip_init(void) static void ip_destroy(void *unused __unused) { - struct ifnet *ifp; int error; #ifdef RSS @@ -405,10 +404,7 @@ ip_destroy(void *unused __unused) in_ifscrub_all(); /* Make sure the IPv4 routes are gone as well. */ - IFNET_RLOCK(); - CK_STAILQ_FOREACH(ifp, &V_ifnet, if_link) - rt_flushifroutes_af(ifp, AF_INET); - IFNET_RUNLOCK(); + rib_flush_routes_family(AF_INET); /* Destroy IP reassembly queue. */ ipreass_destroy(); diff --git a/sys/netinet6/ip6_input.c b/sys/netinet6/ip6_input.c index 80e5acc62548..9ea578f88417 100644 --- a/sys/netinet6/ip6_input.c +++ b/sys/netinet6/ip6_input.c @@ -386,11 +386,12 @@ ip6_destroy(void *unused __unused) /* IF_ADDR_UNLOCK(ifp); */ in6_ifdetach_destroy(ifp); mld_domifdetach(ifp); - /* Make sure any routes are gone as well. */ - rt_flushifroutes_af(ifp, AF_INET6); } IFNET_RUNLOCK(); + /* Make sure any routes are gone as well. */ + rib_flush_routes_family(AF_INET6); + frag6_destroy(); nd6_destroy(); in6_ifattach_destroy();