From owner-dev-commits-src-all@freebsd.org Tue Sep 14 03:39:27 2021 Return-Path: Delivered-To: dev-commits-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 83A31660335 for ; Tue, 14 Sep 2021 03:39:27 +0000 (UTC) (envelope-from wma@semihalf.com) Received: from mail-yb1-xb32.google.com (mail-yb1-xb32.google.com [IPv6:2607:f8b0:4864:20::b32]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4H7pxW2Q8Vz3GZC for ; Tue, 14 Sep 2021 03:39:27 +0000 (UTC) (envelope-from wma@semihalf.com) Received: by mail-yb1-xb32.google.com with SMTP id i12so24814533ybq.9 for ; Mon, 13 Sep 2021 20:39:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=otQrQW8V9U2Pw2OfQW78sPF84cpbU3v3gQYpi46BB3k=; b=fERXj0YHGIRak+mAyFlNH8Mvl9CrWiry8LT5fAg1jIkMIcyITUppMCfhsbKdjJfhxF jf9e69Fm2lH3Hq1v4isrNZwOJOFoAKBGH8HNFlsr1g4z/biLVlkGHr2vnyQuCHwNmv+h nl9nu1iHW7dgvs8V0no6DWX9Uj4NpGT2gtGKe9hA1B1jdcFMljPM6zNMcYOUlW52KJfY Tnpq3Zd6DEaJ1deDZ2L2DWKLw1pYHIjB+CQcZoP7QP54+i9FPCfSRtYLcPTSaAjOIKCi Csg7in2ZAeMZ+KcmoKSi6LISbrxkWJEkiyYkigiX8/dQo+wq2/s+/1ZSGLTe1I1Nq7LJ MeuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=otQrQW8V9U2Pw2OfQW78sPF84cpbU3v3gQYpi46BB3k=; b=HjkJwrjxwySYSOgra9lGOUZcJEg91X/JvTVZu4pffME9CEoxlho+wgl+s9KlD12Mnp qIcY4PIG3HbHuFgdGwrFU8GnRe/mKKtB4YkDP3x7tHpMRnU0NBQ58Ssx3skJUG67d2I+ zpGfB9QsuoPuq8OsXAMnvV4BQA0+32sj8q2f93J/NQn1N5cOnfyxYPG9eM293yquFJkM qUVYzpxaWIHRF0usMJBgkY/7IAkANTW87YkdZk6QU0U8FABN2Qg1Pd1pM07JuNDqw/w9 LA7yNkEqAUQNyPG1742GKr3KIyianCBoT0KdwjJFS2qLw4HOxEjnCKutdbgK97dMOl6n +AFw== X-Gm-Message-State: AOAM530oU3IyGypC3O5rSho61PRKUM6aw8xBYoldbvVssrqcf0YRlpMr UySHPkZL7vG1ef3/0/HtJJH4AqzKsmISMRQTnaiTXQ== X-Google-Smtp-Source: ABdhPJwuSpB1/NdAIQJAOAkMxx5Ex1p2FH/uollfXhG2VJn7qtrD8MGk+p2O7usasOBi9hzZscy3mSxTIsh9vkrCifc= X-Received: by 2002:a25:7ec3:: with SMTP id z186mr18852918ybc.147.1631590766806; Mon, 13 Sep 2021 20:39:26 -0700 (PDT) MIME-Version: 1.0 References: <202109130411.18D4BUs3004805@gitrepo.freebsd.org> In-Reply-To: From: Wojciech Macek Date: Tue, 14 Sep 2021 05:39:15 +0200 Message-ID: Subject: Re: git: 6fa041d7f125 - main - Measure latency of PMC interruptions To: Mitchell Horne Cc: Wojciech Macek , src-committers , dev-commits-src-all@freebsd.org, dev-commits-src-main@freebsd.org X-Rspamd-Queue-Id: 4H7pxW2Q8Vz3GZC X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: dev-commits-src-all@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commit messages for all branches of the src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Sep 2021 03:39:27 -0000 Hi, Thanks for comments, I'll handle all of them as well as docs. Wojtek pon., 13 wrz 2021 o 22:06 Mitchell Horne napisa=C5=82(= a): > On Mon, Sep 13, 2021 at 1:11 AM Wojciech Macek wrote: > > > > The branch main has been updated by wma: > > > > URL: > https://cgit.FreeBSD.org/src/commit/?id=3D6fa041d7f125db65f400af7f520a41f= f78d19cd7 > > > > commit 6fa041d7f125db65f400af7f520a41ff78d19cd7 > > Author: Wojciech Macek > > AuthorDate: 2021-09-13 04:08:32 +0000 > > Commit: Wojciech Macek > > CommitDate: 2021-09-13 04:08:32 +0000 > > > > Measure latency of PMC interruptions > > > > Add HWPMC events to measure latency. > > Provide sysctl to choose the number of outstanding events which > > trigger HWPMC event. > > > > Obtained from: Semihalf > > Sponsored by: Stormshield > > Differential revision: https://reviews.freebsd.org/D31283 > > --- > > sys/kern/kern_intr.c | 58 > ++++++++++++++++++++++++++++++++++++++++++++++------ > > 1 file changed, 52 insertions(+), 6 deletions(-) > > > > diff --git a/sys/kern/kern_intr.c b/sys/kern/kern_intr.c > > index 1660414a50ef..19401877dfbd 100644 > > --- a/sys/kern/kern_intr.c > > +++ b/sys/kern/kern_intr.c > > @@ -30,6 +30,7 @@ > > __FBSDID("$FreeBSD$"); > > > > #include "opt_ddb.h" > > +#include "opt_hwpmc_hooks.h" > > #include "opt_kstack_usage_prof.h" > > > > #include > > @@ -75,6 +76,7 @@ struct intr_thread { > > struct thread *it_thread; /* Kernel thread. */ > > int it_flags; /* (j) IT_* flags. */ > > int it_need; /* Needs service. */ > > + int it_waiting; /* Waiting in the runq. */ > > }; > > > > /* Interrupt thread flags kept in it_flags */ > > @@ -100,13 +102,19 @@ SYSCTL_INT(_hw, OID_AUTO, intr_storm_threshold, > CTLFLAG_RWTUN, > > static int intr_epoch_batch =3D 1000; > > SYSCTL_INT(_hw, OID_AUTO, intr_epoch_batch, CTLFLAG_RWTUN, > &intr_epoch_batch, > > 0, "Maximum interrupt handler executions without re-entering > epoch(9)"); > > +#ifdef HWPMC_HOOKS > > +static int intr_hwpmc_waiting_report_threshold =3D 1; > > +SYSCTL_INT(_hw, OID_AUTO, intr_hwpmc_waiting_report_threshold, > CTLFLAG_RWTUN, > > + &intr_hwpmc_waiting_report_threshold, 1, > > + "Threshold for reporting number of events in a workq"); > > +#endif > > static TAILQ_HEAD(, intr_event) event_list =3D > > TAILQ_HEAD_INITIALIZER(event_list); > > static struct mtx event_lock; > > MTX_SYSINIT(intr_event_list, &event_lock, "intr event list", MTX_DEF); > > > > static void intr_event_update(struct intr_event *ie); > > -static int intr_event_schedule_thread(struct intr_event *ie); > > +static int intr_event_schedule_thread(struct intr_event *ie, struc= t > trapframe *frame); > > static struct intr_thread *ithread_create(const char *name); > > static void ithread_destroy(struct intr_thread *ithread); > > static void ithread_execute_handlers(struct proc *p, > > @@ -115,6 +123,16 @@ static void ithread_loop(void *); > > static void ithread_update(struct intr_thread *ithd); > > static void start_softintr(void *); > > > > +#ifdef HWPMC_HOOKS > > +#include > > +PMC_SOFT_DEFINE( , , intr, all); > > +PMC_SOFT_DEFINE( , , intr, ithread); > > +PMC_SOFT_DEFINE( , , intr, filter); > > +PMC_SOFT_DEFINE( , , intr, stray); > > +PMC_SOFT_DEFINE( , , intr, schedule); > > +PMC_SOFT_DEFINE( , , intr, waiting); > > +#endif > > + > > Hi Wojciech, > > Do you plan to document these new events in the pmc.soft(3) man page? > > Cheers, > Mitchell > > > > /* Map an interrupt type to an ithread priority. */ > > u_char > > intr_priority(enum intr_type flags) > > @@ -773,7 +791,7 @@ intr_handler_barrier(struct intr_handler *handler) > > } > > if ((handler->ih_flags & IH_CHANGED) =3D=3D 0) { > > handler->ih_flags |=3D IH_CHANGED; > > - intr_event_schedule_thread(ie); > > + intr_event_schedule_thread(ie, NULL); > > } > > while ((handler->ih_flags & IH_CHANGED) !=3D 0) > > msleep(handler, &ie->ie_lock, 0, "ih_barr", 0); > > @@ -872,7 +890,7 @@ intr_event_remove_handler(void *cookie) > > KASSERT((handler->ih_flags & IH_DEAD) =3D=3D 0, > > ("duplicate handle remove")); > > handler->ih_flags |=3D IH_DEAD; > > - intr_event_schedule_thread(ie); > > + intr_event_schedule_thread(ie, NULL); > > while (handler->ih_flags & IH_DEAD) > > msleep(handler, &ie->ie_lock, 0, "iev_rmh", 0); > > intr_event_update(ie); > > @@ -944,7 +962,7 @@ intr_event_resume_handler(void *cookie) > > } > > > > static int > > -intr_event_schedule_thread(struct intr_event *ie) > > +intr_event_schedule_thread(struct intr_event *ie, struct trapframe > *frame) > > { > > struct intr_entropy entropy; > > struct intr_thread *it; > > @@ -986,11 +1004,28 @@ intr_event_schedule_thread(struct intr_event *ie= ) > > atomic_store_rel_int(&it->it_need, 1); > > thread_lock(td); > > if (TD_AWAITING_INTR(td)) { > > +#ifdef HWPMC_HOOKS > > + atomic_set_int(&it->it_waiting, 0); > > + if (frame !=3D NULL) > > + PMC_SOFT_CALL_TF( , , intr, schedule, frame); > > + else > > + PMC_SOFT_CALL( , , intr, schedule); > > +#endif > > CTR3(KTR_INTR, "%s: schedule pid %d (%s)", __func__, > td->td_proc->p_pid, > > td->td_name); > > TD_CLR_IWAIT(td); > > sched_add(td, SRQ_INTR); > > } else { > > +#ifdef HWPMC_HOOKS > > + atomic_add_int(&it->it_waiting, 1); > > + > > + if (atomic_load_int(&it->it_waiting) >=3D > intr_hwpmc_waiting_report_threshold) { > > + if (frame !=3D NULL) > > + PMC_SOFT_CALL_TF( , , intr, waiting, > frame); > > + else > > + PMC_SOFT_CALL( , , intr, waiting); > > + } > > +#endif > > CTR5(KTR_INTR, "%s: pid %d (%s): it_need %d, state %d", > > __func__, td->td_proc->p_pid, td->td_name, > it->it_need, TD_GET_STATE(td)); > > thread_unlock(td); > > @@ -1083,7 +1118,7 @@ swi_sched(void *cookie, int flags) > > #endif > > } else { > > VM_CNT_INC(v_soft); > > - error =3D intr_event_schedule_thread(ie); > > + error =3D intr_event_schedule_thread(ie, NULL); > > KASSERT(error =3D=3D 0, ("stray software interrupt")); > > } > > } > > @@ -1374,12 +1409,23 @@ intr_event_handle(struct intr_event *ie, struct > trapframe *frame) > > ret =3D ih->ih_filter(frame); > > else > > ret =3D ih->ih_filter(ih->ih_argument); > > +#ifdef HWPMC_HOOKS > > + PMC_SOFT_CALL_TF( , , intr, all, frame); > > +#endif > > KASSERT(ret =3D=3D FILTER_STRAY || > > ((ret & (FILTER_SCHEDULE_THREAD | FILTER_HANDLED)) > !=3D 0 && > > (ret & ~(FILTER_SCHEDULE_THREAD | FILTER_HANDLED)) > =3D=3D 0), > > ("%s: incorrect return value %#x from %s", __func__= , > ret, > > ih->ih_name)); > > filter =3D filter || ret =3D=3D FILTER_HANDLED; > > +#ifdef HWPMC_HOOKS > > + if (ret & FILTER_SCHEDULE_THREAD) > > + PMC_SOFT_CALL_TF( , , intr, ithread, frame); > > + else if (ret & FILTER_HANDLED) > > + PMC_SOFT_CALL_TF( , , intr, filter, frame); > > + else if (ret =3D=3D FILTER_STRAY) > > + PMC_SOFT_CALL_TF( , , intr, stray, frame); > > +#endif > > > > /* > > * Wrapper handler special handling: > > @@ -1416,7 +1462,7 @@ intr_event_handle(struct intr_event *ie, struct > trapframe *frame) > > if (thread) { > > int error __unused; > > > > - error =3D intr_event_schedule_thread(ie); > > + error =3D intr_event_schedule_thread(ie, frame); > > KASSERT(error =3D=3D 0, ("bad stray interrupt")); > > } > > critical_exit(); >