Date: Mon, 2 Sep 2024 09:12:50 GMT From: Andrew Turner <andrew@FreeBSD.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org Subject: git: 39757e10689d - stable/13 - buf_ring: Ensure correct ordering of loads Message-ID: <202409020912.4829Cort050083@gitrepo.freebsd.org>
next in thread | raw e-mail | index | archive | help
The branch stable/13 has been updated by andrew: URL: https://cgit.FreeBSD.org/src/commit/?id=39757e10689d44036b5d10de41d766641e59eec0 commit 39757e10689d44036b5d10de41d766641e59eec0 Author: Andrew Turner <andrew@FreeBSD.org> AuthorDate: 2024-08-19 09:07:26 +0000 Commit: Andrew Turner <andrew@FreeBSD.org> CommitDate: 2024-09-02 09:11:57 +0000 buf_ring: Ensure correct ordering of loads When enqueueing on an architecture with a weak memory model ensure loading br->br_prod_head and br->br_cons_tail are ordered correctly. If br_cons_tail is loaded first then other threads may perform a dequeue and enqueue before br_prod_head is loaded. This will mean the tail is one less than it should be and the code under the prod_next == cons_tail check could incorrectly be skipped. buf_ring_dequeue_mc has the same issue with br->br_prod_tail and br->br_cons_head so needs the same fix. Reported by: Ali Saidi <alisaidi@amazon.com> Co-developed by: Ali Saidi <alisaidi@amazon.com> Reviewed by: imp, kib, markj Sponsored by: Arm Ltd Differential Revision: https://reviews.freebsd.org/D46155 (cherry picked from commit fe2445f47d027c73aa7266669e7d94b70d3949a4) --- sys/sys/buf_ring.h | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/sys/sys/buf_ring.h b/sys/sys/buf_ring.h index 66e1e55bc5e9..512f20dc13e2 100644 --- a/sys/sys/buf_ring.h +++ b/sys/sys/buf_ring.h @@ -89,7 +89,17 @@ buf_ring_enqueue(struct buf_ring *br, void *buf) #endif critical_enter(); do { - prod_head = br->br_prod_head; + /* + * br->br_prod_head needs to be read before br->br_cons_tail. + * If not then we could perform the dequeue and enqueue + * between reading br_cons_tail and reading br_prod_head. This + * could give us values where br_cons_head == br_prod_tail + * (after masking). + * + * To work around this us a load acquire. This is just to + * ensure ordering within this thread. + */ + prod_head = atomic_load_acq_32(&br->br_prod_head); prod_next = prod_head + 1; cons_tail = atomic_load_acq_32(&br->br_cons_tail); @@ -137,7 +147,12 @@ buf_ring_dequeue_mc(struct buf_ring *br) critical_enter(); mask = br->br_cons_mask; do { - cons_head = br->br_cons_head; + /* + * As with buf_ring_enqueue ensure we read the head before + * the tail. If we read them in the wrong order we may + * think the bug_ring is full when it is empty. + */ + cons_head = atomic_load_acq_32(&br->br_cons_head); cons_next = cons_head + 1; prod_tail = atomic_load_acq_32(&br->br_prod_tail);
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?202409020912.4829Cort050083>