From owner-svn-src-stable@freebsd.org Thu Oct 18 04:36:28 2018 Return-Path: Delivered-To: svn-src-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D338310C9A8F; Thu, 18 Oct 2018 04:36:27 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8894A79E1D; Thu, 18 Oct 2018 04:36:27 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7F54F1AFE6; Thu, 18 Oct 2018 04:36:27 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w9I4aRZk010596; Thu, 18 Oct 2018 04:36:27 GMT (envelope-from kp@FreeBSD.org) Received: (from kp@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w9I4aPTR010587; Thu, 18 Oct 2018 04:36:25 GMT (envelope-from kp@FreeBSD.org) Message-Id: <201810180436.w9I4aPTR010587@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kp set sender to kp@FreeBSD.org using -f From: Kristof Provost Date: Thu, 18 Oct 2018 04:36:25 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r339418 - in stable/11/sys: net netpfil/pf X-SVN-Group: stable-11 X-SVN-Commit-Author: kp X-SVN-Commit-Paths: in stable/11/sys: net netpfil/pf X-SVN-Commit-Revision: 339418 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-stable@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: SVN commit messages for all the -stable branches of the src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Oct 2018 04:36:28 -0000 Author: kp Date: Thu Oct 18 04:36:25 2018 New Revision: 339418 URL: https://svnweb.freebsd.org/changeset/base/339418 Log: MFC r334375, r334379: pf: Replace rwlock on PF_RULES_LOCK with rmlock Given that PF_RULES_LOCK is a mostly read lock, replace the rwlock with rmlock. This change improves packet processing rate in high pps environments. Benchmarking by olivier@ shows a 65% improvement in pps. While here, also eliminate all appearances of "sys/rwlock.h" includes since it is not used anymore. Submitted by: farrokhi@ Modified: stable/11/sys/net/pfvar.h stable/11/sys/netpfil/pf/if_pfsync.c stable/11/sys/netpfil/pf/pf.c stable/11/sys/netpfil/pf/pf_if.c stable/11/sys/netpfil/pf/pf_ioctl.c stable/11/sys/netpfil/pf/pf_lb.c stable/11/sys/netpfil/pf/pf_norm.c stable/11/sys/netpfil/pf/pf_osfp.c stable/11/sys/netpfil/pf/pf_table.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/net/pfvar.h ============================================================================== --- stable/11/sys/net/pfvar.h Thu Oct 18 02:07:30 2018 (r339417) +++ stable/11/sys/net/pfvar.h Thu Oct 18 04:36:25 2018 (r339418) @@ -36,8 +36,11 @@ #include #include #include +#include #include #include +#include +#include #include #include @@ -145,14 +148,15 @@ extern struct mtx pf_unlnkdrules_mtx; #define PF_UNLNKDRULES_LOCK() mtx_lock(&pf_unlnkdrules_mtx) #define PF_UNLNKDRULES_UNLOCK() mtx_unlock(&pf_unlnkdrules_mtx) -extern struct rwlock pf_rules_lock; -#define PF_RULES_RLOCK() rw_rlock(&pf_rules_lock) -#define PF_RULES_RUNLOCK() rw_runlock(&pf_rules_lock) -#define PF_RULES_WLOCK() rw_wlock(&pf_rules_lock) -#define PF_RULES_WUNLOCK() rw_wunlock(&pf_rules_lock) -#define PF_RULES_ASSERT() rw_assert(&pf_rules_lock, RA_LOCKED) -#define PF_RULES_RASSERT() rw_assert(&pf_rules_lock, RA_RLOCKED) -#define PF_RULES_WASSERT() rw_assert(&pf_rules_lock, RA_WLOCKED) +extern struct rmlock pf_rules_lock; +#define PF_RULES_RLOCK_TRACKER struct rm_priotracker _pf_rules_tracker +#define PF_RULES_RLOCK() rm_rlock(&pf_rules_lock, &_pf_rules_tracker) +#define PF_RULES_RUNLOCK() rm_runlock(&pf_rules_lock, &_pf_rules_tracker) +#define PF_RULES_WLOCK() rm_wlock(&pf_rules_lock) +#define PF_RULES_WUNLOCK() rm_wunlock(&pf_rules_lock) +#define PF_RULES_ASSERT() rm_assert(&pf_rules_lock, RA_LOCKED) +#define PF_RULES_RASSERT() rm_assert(&pf_rules_lock, RA_RLOCKED) +#define PF_RULES_WASSERT() rm_assert(&pf_rules_lock, RA_WLOCKED) #define PF_MODVER 1 #define PFLOG_MODVER 1 Modified: stable/11/sys/netpfil/pf/if_pfsync.c ============================================================================== --- stable/11/sys/netpfil/pf/if_pfsync.c Thu Oct 18 02:07:30 2018 (r339417) +++ stable/11/sys/netpfil/pf/if_pfsync.c Thu Oct 18 04:36:25 2018 (r339418) @@ -585,6 +585,8 @@ pfsync_input(struct mbuf **mp, int *offp __unused, int int rv; uint16_t count; + PF_RULES_RLOCK_TRACKER; + *mp = NULL; V_pfsyncstats.pfsyncs_ipackets++; Modified: stable/11/sys/netpfil/pf/pf.c ============================================================================== --- stable/11/sys/netpfil/pf/pf.c Thu Oct 18 02:07:30 2018 (r339417) +++ stable/11/sys/netpfil/pf/pf.c Thu Oct 18 04:36:25 2018 (r339418) @@ -1454,9 +1454,11 @@ pf_purge_thread(void *unused __unused) VNET_ITERATOR_DECL(vnet_iter); u_int idx = 0; + PF_RULES_RLOCK_TRACKER; + for (;;) { PF_RULES_RLOCK(); - rw_sleep(pf_purge_thread, &pf_rules_lock, 0, "pftm", hz / 10); + rm_sleep(pf_purge_thread, &pf_rules_lock, 0, "pftm", hz / 10); PF_RULES_RUNLOCK(); VNET_LIST_RLOCK(); @@ -5908,6 +5910,8 @@ pf_test(int dir, int pflags, struct ifnet *ifp, struct struct pf_pdesc pd; int off, dirndx, pqid = 0; + PF_RULES_RLOCK_TRACKER; + M_ASSERTPKTHDR(m); if (!V_pf_status.running) @@ -6295,6 +6299,7 @@ pf_test6(int dir, int pflags, struct ifnet *ifp, struc struct pf_pdesc pd; int off, terminal = 0, dirndx, rh_cnt = 0, pqid = 0; + PF_RULES_RLOCK_TRACKER; M_ASSERTPKTHDR(m); if (!V_pf_status.running) Modified: stable/11/sys/netpfil/pf/pf_if.c ============================================================================== --- stable/11/sys/netpfil/pf/pf_if.c Thu Oct 18 02:07:30 2018 (r339417) +++ stable/11/sys/netpfil/pf/pf_if.c Thu Oct 18 04:36:25 2018 (r339418) @@ -44,7 +44,6 @@ __FBSDID("$FreeBSD$"); #include #include #include -#include #include #include Modified: stable/11/sys/netpfil/pf/pf_ioctl.c ============================================================================== --- stable/11/sys/netpfil/pf/pf_ioctl.c Thu Oct 18 02:07:30 2018 (r339417) +++ stable/11/sys/netpfil/pf/pf_ioctl.c Thu Oct 18 04:36:25 2018 (r339418) @@ -57,7 +57,6 @@ __FBSDID("$FreeBSD$"); #include #include #include -#include #include #include #include @@ -201,7 +200,7 @@ VNET_DEFINE(int, pf_vnet_active); int pf_end_threads; -struct rwlock pf_rules_lock; +struct rmlock pf_rules_lock; struct sx pf_ioctl_lock; /* pfsync */ @@ -991,6 +990,7 @@ static int pfioctl(struct cdev *dev, u_long cmd, caddr_t addr, int flags, struct thread *td) { int error = 0; + PF_RULES_RLOCK_TRACKER; /* XXX keep in sync with switch() below */ if (securelevel_gt(td->td_ucred, 2)) @@ -3912,7 +3912,7 @@ pf_load(void) { int error; - rw_init(&pf_rules_lock, "pf rulesets"); + rm_init(&pf_rules_lock, "pf rulesets"); sx_init(&pf_ioctl_lock, "pf ioctl"); pf_mtag_initialize(); @@ -3976,7 +3976,7 @@ pf_unload(void) pf_end_threads = 1; while (pf_end_threads < 2) { wakeup_one(pf_purge_thread); - rw_sleep(pf_purge_thread, &pf_rules_lock, 0, "pftmo", 0); + rm_sleep(pf_purge_thread, &pf_rules_lock, 0, "pftmo", 0); } if (pf_dev != NULL) @@ -3984,7 +3984,7 @@ pf_unload(void) pfi_cleanup(); - rw_destroy(&pf_rules_lock); + rm_destroy(&pf_rules_lock); sx_destroy(&pf_ioctl_lock); return (error); Modified: stable/11/sys/netpfil/pf/pf_lb.c ============================================================================== --- stable/11/sys/netpfil/pf/pf_lb.c Thu Oct 18 02:07:30 2018 (r339417) +++ stable/11/sys/netpfil/pf/pf_lb.c Thu Oct 18 04:36:25 2018 (r339418) @@ -44,7 +44,6 @@ __FBSDID("$FreeBSD$"); #include #include #include -#include #include #include Modified: stable/11/sys/netpfil/pf/pf_norm.c ============================================================================== --- stable/11/sys/netpfil/pf/pf_norm.c Thu Oct 18 02:07:30 2018 (r339417) +++ stable/11/sys/netpfil/pf/pf_norm.c Thu Oct 18 04:36:25 2018 (r339418) @@ -39,7 +39,6 @@ __FBSDID("$FreeBSD$"); #include #include #include -#include #include #include Modified: stable/11/sys/netpfil/pf/pf_osfp.c ============================================================================== --- stable/11/sys/netpfil/pf/pf_osfp.c Thu Oct 18 02:07:30 2018 (r339417) +++ stable/11/sys/netpfil/pf/pf_osfp.c Thu Oct 18 04:36:25 2018 (r339418) @@ -25,7 +25,6 @@ __FBSDID("$FreeBSD$"); #include #include #include -#include #include #include Modified: stable/11/sys/netpfil/pf/pf_table.c ============================================================================== --- stable/11/sys/netpfil/pf/pf_table.c Thu Oct 18 02:07:30 2018 (r339417) +++ stable/11/sys/netpfil/pf/pf_table.c Thu Oct 18 04:36:25 2018 (r339418) @@ -42,7 +42,6 @@ __FBSDID("$FreeBSD$"); #include #include #include -#include #include #include