Date: Wed, 29 Jul 2026 04:05:43 -0700 From: Kevin Bowling <kevin.bowling@kev009.com> To: Mark Johnston <markj@freebsd.org> Cc: src-committers@freebsd.org, dev-commits-src-all@freebsd.org, dev-commits-src-main@freebsd.org Subject: Re: git: 1f4b0ea4f3eb - main - kqueue: Add a helper macro for sleeping on in-flux knotes Message-ID: <CAK7dMtDU8F=VAusGYYkEfAvxtRO%2BVB5MohjX=-sC1qbP_YE=mQ@mail.gmail.com> In-Reply-To: <6a67e66f.32351.45a31bd3@gitrepo.freebsd.org>
index | next in thread | previous in thread | raw e-mail
Hi Mark, This is resulting in relatively easy to trigger panic with 'syncthing' running under INVARIANTS: panic: kqueue_scan: knote 0xfffffe018aeab930 not on kqueue 0xfffff80010aa2400 The panic occurs at the new KASSERT in KQ_FLUX_SLEEP_WMESG(), called from kqueue_scan() line 2197. Two Syncthing threads were concurrently scanning the same kqueue. kgdb showed that the knote named in the panic was exactly the other scanning thread’s local marker: &marker = 0xfffffe018aeab930 marker.kn_status = KN_MARKER marker.kn_kq = NULL kq = 0xfffff80010aa2400 I have the full backtraces if useful. Regards, Kevin On Mon, Jul 27, 2026 at 4:15 PM Mark Johnston <markj@freebsd.org> wrote: > > The branch main has been updated by markj: > > URL: https://cgit.FreeBSD.org/src/commit/?id=1f4b0ea4f3eb1b8a885eff8bd0d332156f0c3e1f > > commit 1f4b0ea4f3eb1b8a885eff8bd0d332156f0c3e1f > Author: Mark Johnston <markj@FreeBSD.org> > AuthorDate: 2026-07-27 23:12:16 +0000 > Commit: Mark Johnston <markj@FreeBSD.org> > CommitDate: 2026-07-27 23:12:16 +0000 > > kqueue: Add a helper macro for sleeping on in-flux knotes > > Other in-flux operations are implemented by this set of macros, so we > should do the same for sleeping. > > No functional change intended. > > Reviewed by: kib > MFC after: 1 week > Sponsored by: The FreeBSD Foundation > Differential Revision: https://reviews.freebsd.org/D58443 > --- > sys/kern/kern_event.c | 32 ++++++++++++++++---------------- > 1 file changed, 16 insertions(+), 16 deletions(-) > > diff --git a/sys/kern/kern_event.c b/sys/kern/kern_event.c > index 23e9d309f74c..60c3b18a07eb 100644 > --- a/sys/kern/kern_event.c > +++ b/sys/kern/kern_event.c > @@ -257,6 +257,14 @@ SYSCTL_UINT(_kern, OID_AUTO, kq_calloutmax, CTLFLAG_RW, > wakeup((kq)); \ > } \ > } while (0) > +#define KQ_FLUX_SLEEP_WMESG(kq, kn, flags, wmesg) do { \ > + KASSERT((kn)->kn_kq == (kq), \ > + ("%s: knote %p not on kqueue %p", __func__, kn, kq)); \ > + (kq)->kq_state |= KQ_FLUXWAIT; \ > + msleep((kq), &(kq)->kq_lock, PSOCK | (flags), (wmesg), 0); \ > +} while (0) > +#define KQ_FLUX_SLEEP(kq, kn, flags) \ > + KQ_FLUX_SLEEP_WMESG(kq, kn, flags, "kqfluxwt") > #define KQ_UNLOCK_FLUX(kq) do { \ > KQ_FLUX_WAKEUP(kq); \ > mtx_unlock(&(kq)->kq_lock); \ > @@ -1789,8 +1797,7 @@ findkn: > FILEDESC_XUNLOCK(td->td_proc->p_fd); > filedesc_unlock = 0; > } > - kq->kq_state |= KQ_FLUXWAIT; > - msleep(kq, &kq->kq_lock, PSOCK | PDROP, "kqflxwt", 0); > + KQ_FLUX_SLEEP(kq, kn, PDROP); > if (fp != NULL) { > fdrop(fp, td); > fp = NULL; > @@ -2187,9 +2194,7 @@ retry: > influx = 0; > KQ_FLUX_WAKEUP(kq); > } > - kq->kq_state |= KQ_FLUXWAIT; > - error = msleep(kq, &kq->kq_lock, PSOCK, > - "kqflxwt", 0); > + KQ_FLUX_SLEEP(kq, kn, 0); > continue; > } > > @@ -2423,8 +2428,7 @@ kqueue_drain(struct kqueue *kq, struct thread *td) > for (i = 0; i < kq->kq_knlistsize; i++) { > while ((kn = SLIST_FIRST(&kq->kq_knlist[i])) != NULL) { > if (kn_in_flux(kn)) { > - kq->kq_state |= KQ_FLUXWAIT; > - msleep(kq, &kq->kq_lock, PSOCK, "kqclo1", 0); > + KQ_FLUX_SLEEP_WMESG(kq, kn, 0, "kqclo1"); > continue; > } > kn_enter_flux(kn); > @@ -2437,9 +2441,8 @@ kqueue_drain(struct kqueue *kq, struct thread *td) > for (i = 0; i <= kq->kq_knhashmask; i++) { > while ((kn = SLIST_FIRST(&kq->kq_knhash[i])) != NULL) { > if (kn_in_flux(kn)) { > - kq->kq_state |= KQ_FLUXWAIT; > - msleep(kq, &kq->kq_lock, PSOCK, > - "kqclo2", 0); > + KQ_FLUX_SLEEP_WMESG(kq, kn, 0, > + "kqclo2"); > continue; > } > kn_enter_flux(kn); > @@ -2834,8 +2837,7 @@ knlist_cleardel(struct knlist *knl, struct thread *td, int islocked, int killkn) > KQ_LOCK(kq); > KASSERT(kn_in_flux(kn), ("knote removed w/o list lock")); > knl->kl_unlock(knl->kl_lockarg); > - kq->kq_state |= KQ_FLUXWAIT; > - msleep(kq, &kq->kq_lock, PSOCK | PDROP, "kqkclr", 0); > + KQ_FLUX_SLEEP_WMESG(kq, kn, PDROP, "kqkclr"); > kq = NULL; > knl->kl_lock(knl->kl_lockarg); > } > @@ -2881,8 +2883,7 @@ knote_fdclose(struct thread *td, int fd) > * the case that it's in the process of being > * dropped anyways. > */ > - kq->kq_state |= KQ_FLUXWAIT; > - msleep(kq, &kq->kq_lock, PSOCK, "kqflxwt", 0); > + KQ_FLUX_SLEEP(kq, kn, 0); > continue; > } > kn_enter_flux(kn); > @@ -2946,8 +2947,7 @@ knote_drop_detached(struct knote *kn, struct thread *td) > kn, kn->kn_influx)); > if (kn->kn_influx == 1) > break; > - kq->kq_state |= KQ_FLUXWAIT; > - msleep(kq, &kq->kq_lock, PSOCK, "kqflxwt", 0); > + KQ_FLUX_SLEEP(kq, kn, 0); > } > > MPASS(kn->kn_kq == kq); >home | help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAK7dMtDU8F=VAusGYYkEfAvxtRO%2BVB5MohjX=-sC1qbP_YE=mQ>