From owner-dev-commits-src-branches@freebsd.org Wed Aug 11 13:25:22 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id EB626653666; Wed, 11 Aug 2021 13:25:22 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Gl9YG451Sz3CjB; Wed, 11 Aug 2021 13:25:22 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 72D6310138; Wed, 11 Aug 2021 13:25:22 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 17BDPMaJ004953; Wed, 11 Aug 2021 13:25:22 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 17BDPMqY004952; Wed, 11 Aug 2021 13:25:22 GMT (envelope-from git) Date: Wed, 11 Aug 2021 13:25:22 GMT Message-Id: <202108111325.17BDPMqY004952@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mateusz Guzik Subject: git: 2522a90dcd41 - stable/12 - pf: batch critical section for several counters MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mjg X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: 2522a90dcd41c9c8326525fa23876bb2acc7d51a Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Aug 2021 13:25:23 -0000 The branch stable/12 has been updated by mjg: URL: https://cgit.FreeBSD.org/src/commit/?id=2522a90dcd41c9c8326525fa23876bb2acc7d51a commit 2522a90dcd41c9c8326525fa23876bb2acc7d51a Author: Mateusz Guzik AuthorDate: 2021-07-24 07:47:40 +0000 Commit: Mateusz Guzik CommitDate: 2021-08-11 12:23:20 +0000 pf: batch critical section for several counters Reviewed by: kp Sponsored by: Rubicon Communications, LLC ("Netgate") (cherry picked from commit 87c010e6e364e96e2c1546b3c2bbcbef1dcd422f) --- sys/netpfil/pf/pf.c | 48 +++++++++++++++++++++++++++-------------------- sys/netpfil/pf/pf_ioctl.c | 12 +++++++----- sys/netpfil/pf/pf_norm.c | 18 ++++++++++++------ 3 files changed, 47 insertions(+), 31 deletions(-) diff --git a/sys/netpfil/pf/pf.c b/sys/netpfil/pf/pf.c index 0c485189ec16..f1b1890f50c5 100644 --- a/sys/netpfil/pf/pf.c +++ b/sys/netpfil/pf/pf.c @@ -3756,8 +3756,10 @@ pf_test_rule(struct pf_krule **rm, struct pf_kstate **sm, int direction, rtableid = r->rtableid; if (r->anchor == NULL) { if (r->action == PF_MATCH) { - pf_counter_u64_add(&r->packets[direction == PF_OUT], 1); - pf_counter_u64_add(&r->bytes[direction == PF_OUT], pd->tot_len); + pf_counter_u64_critical_enter(); + pf_counter_u64_add_protected(&r->packets[direction == PF_OUT], 1); + pf_counter_u64_add_protected(&r->bytes[direction == PF_OUT], pd->tot_len); + pf_counter_u64_critical_exit(); pf_rule_to_actions(r, &pd->act); if (r->log) PFLOG_PACKET(kif, m, af, @@ -4170,8 +4172,10 @@ pf_test_fragment(struct pf_krule **rm, int direction, struct pfi_kkif *kif, else { if (r->anchor == NULL) { if (r->action == PF_MATCH) { - pf_counter_u64_add(&r->packets[direction == PF_OUT], 1); - pf_counter_u64_add(&r->bytes[direction == PF_OUT], pd->tot_len); + pf_counter_u64_critical_enter(); + pf_counter_u64_add_protected(&r->packets[direction == PF_OUT], 1); + pf_counter_u64_add_protected(&r->bytes[direction == PF_OUT], pd->tot_len); + pf_counter_u64_critical_exit(); pf_rule_to_actions(r, &pd->act); if (r->log) PFLOG_PACKET(kif, m, af, @@ -6555,24 +6559,25 @@ done: (s == NULL)); } - pf_counter_u64_add(&kif->pfik_bytes[0][dir == PF_OUT][action != PF_PASS], + pf_counter_u64_critical_enter(); + pf_counter_u64_add_protected(&kif->pfik_bytes[0][dir == PF_OUT][action != PF_PASS], pd.tot_len); - pf_counter_u64_add(&kif->pfik_packets[0][dir == PF_OUT][action != PF_PASS], + pf_counter_u64_add_protected(&kif->pfik_packets[0][dir == PF_OUT][action != PF_PASS], 1); if (action == PF_PASS || r->action == PF_DROP) { dirndx = (dir == PF_OUT); - pf_counter_u64_add(&r->packets[dirndx], 1); - pf_counter_u64_add(&r->bytes[dirndx], pd.tot_len); + pf_counter_u64_add_protected(&r->packets[dirndx], 1); + pf_counter_u64_add_protected(&r->bytes[dirndx], pd.tot_len); if (a != NULL) { - pf_counter_u64_add(&a->packets[dirndx], 1); - pf_counter_u64_add(&a->bytes[dirndx], pd.tot_len); + pf_counter_u64_add_protected(&a->packets[dirndx], 1); + pf_counter_u64_add_protected(&a->bytes[dirndx], pd.tot_len); } if (s != NULL) { if (s->nat_rule.ptr != NULL) { - pf_counter_u64_add(&s->nat_rule.ptr->packets[dirndx], + pf_counter_u64_add_protected(&s->nat_rule.ptr->packets[dirndx], 1); - pf_counter_u64_add(&s->nat_rule.ptr->bytes[dirndx], + pf_counter_u64_add_protected(&s->nat_rule.ptr->bytes[dirndx], pd.tot_len); } if (s->src_node != NULL) { @@ -6610,6 +6615,7 @@ done: pd.af, pd.tot_len, dir == PF_OUT, r->action == PF_PASS, tr->dst.neg); } + pf_counter_u64_critical_exit(); switch (action) { case PF_SYNPROXY_DROP: @@ -6963,24 +6969,25 @@ done: &pd, (s == NULL)); } - pf_counter_u64_add(&kif->pfik_bytes[1][dir == PF_OUT][action != PF_PASS], + pf_counter_u64_critical_enter(); + pf_counter_u64_add_protected(&kif->pfik_bytes[1][dir == PF_OUT][action != PF_PASS], pd.tot_len); - pf_counter_u64_add(&kif->pfik_packets[1][dir == PF_OUT][action != PF_PASS], + pf_counter_u64_add_protected(&kif->pfik_packets[1][dir == PF_OUT][action != PF_PASS], 1); if (action == PF_PASS || r->action == PF_DROP) { dirndx = (dir == PF_OUT); - pf_counter_u64_add(&r->packets[dirndx], 1); - pf_counter_u64_add(&r->bytes[dirndx], pd.tot_len); + pf_counter_u64_add_protected(&r->packets[dirndx], 1); + pf_counter_u64_add_protected(&r->bytes[dirndx], pd.tot_len); if (a != NULL) { - pf_counter_u64_add(&a->packets[dirndx], 1); - pf_counter_u64_add(&a->bytes[dirndx], pd.tot_len); + pf_counter_u64_add_protected(&a->packets[dirndx], 1); + pf_counter_u64_add_protected(&a->bytes[dirndx], pd.tot_len); } if (s != NULL) { if (s->nat_rule.ptr != NULL) { - pf_counter_u64_add(&s->nat_rule.ptr->packets[dirndx], + pf_counter_u64_add_protected(&s->nat_rule.ptr->packets[dirndx], 1); - pf_counter_u64_add(&s->nat_rule.ptr->bytes[dirndx], + pf_counter_u64_add_protected(&s->nat_rule.ptr->bytes[dirndx], pd.tot_len); } if (s->src_node != NULL) { @@ -7016,6 +7023,7 @@ done: pd.af, pd.tot_len, dir == PF_OUT, r->action == PF_PASS, tr->dst.neg); } + pf_counter_u64_critical_exit(); switch (action) { case PF_SYNPROXY_DROP: diff --git a/sys/netpfil/pf/pf_ioctl.c b/sys/netpfil/pf/pf_ioctl.c index 81b50c237252..4f1786b36a30 100644 --- a/sys/netpfil/pf/pf_ioctl.c +++ b/sys/netpfil/pf/pf_ioctl.c @@ -1120,16 +1120,18 @@ pf_commit_rules(u_int32_t ticket, int rs_num, char *anchor) while ((tail != NULL) && ! pf_krule_compare(tail, rule)) tail = TAILQ_NEXT(tail, entries); if (tail != NULL) { - pf_counter_u64_add(&rule->evaluations, + pf_counter_u64_critical_enter(); + pf_counter_u64_add_protected(&rule->evaluations, pf_counter_u64_fetch(&tail->evaluations)); - pf_counter_u64_add(&rule->packets[0], + pf_counter_u64_add_protected(&rule->packets[0], pf_counter_u64_fetch(&tail->packets[0])); - pf_counter_u64_add(&rule->packets[1], + pf_counter_u64_add_protected(&rule->packets[1], pf_counter_u64_fetch(&tail->packets[1])); - pf_counter_u64_add(&rule->bytes[0], + pf_counter_u64_add_protected(&rule->bytes[0], pf_counter_u64_fetch(&tail->bytes[0])); - pf_counter_u64_add(&rule->bytes[1], + pf_counter_u64_add_protected(&rule->bytes[1], pf_counter_u64_fetch(&tail->bytes[1])); + pf_counter_u64_critical_exit(); } } } diff --git a/sys/netpfil/pf/pf_norm.c b/sys/netpfil/pf/pf_norm.c index 19d3a268e4f8..40e4b1bae2f9 100644 --- a/sys/netpfil/pf/pf_norm.c +++ b/sys/netpfil/pf/pf_norm.c @@ -1062,8 +1062,10 @@ pf_normalize_ip(struct mbuf **m0, int dir, struct pfi_kkif *kif, u_short *reason if (r == NULL || r->action == PF_NOSCRUB) return (PF_PASS); - pf_counter_u64_add(&r->packets[dir == PF_OUT], 1); - pf_counter_u64_add(&r->bytes[dir == PF_OUT], pd->tot_len); + pf_counter_u64_critical_enter(); + pf_counter_u64_add_protected(&r->packets[dir == PF_OUT], 1); + pf_counter_u64_add_protected(&r->bytes[dir == PF_OUT], pd->tot_len); + pf_counter_u64_critical_exit(); /* Check for illegal packets */ if (hlen < (int)sizeof(struct ip)) { @@ -1204,8 +1206,10 @@ pf_normalize_ip6(struct mbuf **m0, int dir, struct pfi_kkif *kif, if (r == NULL || r->action == PF_NOSCRUB) return (PF_PASS); - pf_counter_u64_add(&r->packets[dir == PF_OUT], 1); - pf_counter_u64_add(&r->bytes[dir == PF_OUT], pd->tot_len); + pf_counter_u64_critical_enter(); + pf_counter_u64_add_protected(&r->packets[dir == PF_OUT], 1); + pf_counter_u64_add_protected(&r->bytes[dir == PF_OUT], pd->tot_len); + pf_counter_u64_critical_exit(); /* Check for illegal packets */ if (sizeof(struct ip6_hdr) + IPV6_MAXPACKET < m->m_pkthdr.len) @@ -1389,8 +1393,10 @@ pf_normalize_tcp(int dir, struct pfi_kkif *kif, struct mbuf *m, int ipoff, if (rm == NULL || rm->action == PF_NOSCRUB) return (PF_PASS); - pf_counter_u64_add(&r->packets[dir == PF_OUT], 1); - pf_counter_u64_add(&r->bytes[dir == PF_OUT], pd->tot_len); + pf_counter_u64_critical_enter(); + pf_counter_u64_add_protected(&r->packets[dir == PF_OUT], 1); + pf_counter_u64_add_protected(&r->bytes[dir == PF_OUT], pd->tot_len); + pf_counter_u64_critical_exit(); if (rm->rule_flag & PFRULE_REASSEMBLE_TCP) pd->flags |= PFDESC_TCP_NORM;