From owner-dev-commits-src-all@freebsd.org Fri Sep 17 16:44:52 2021 Return-Path: Delivered-To: dev-commits-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5DE0C6B59A7; Fri, 17 Sep 2021 16:44:52 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: from mail-lf1-x12d.google.com (mail-lf1-x12d.google.com [IPv6:2a00:1450:4864:20::12d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HB0DM1vNNz4Sbf; Fri, 17 Sep 2021 16:44:51 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: by mail-lf1-x12d.google.com with SMTP id d42so13672350lfv.10; Fri, 17 Sep 2021 09:44:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=AVSDB95rilSPFCbqIS7Vwa2mE5UN+mtkV1Ti5BvLwd0=; b=EsY4r5itpv0rs3BVUkkTPj7oj8/HHowOVt430PQsHwsVN50eVrd1vlfqBRMiW45nD9 /kEFN5tZHuMMYXf8WGc/PwcBu5JnJIkMVdgJPZyv4/MDa1lx8VKXL3qRWNU1+drDwoGH KRHaUYl73G0NWgvu5PrdfUkFJHj2DA+ygcT5BeFgRwGlPPBwBLEySmh4v1t1LQsUPof7 0Uzo89Oh5UakRfKxT7GM+HGDc0VHvLSB4smydcAFUP8TngFF5sqNBfxdBLpOVRVeX066 5i+TVriQ428qzhAypjB8tkkbFv244Xcks6pdj/Be3NIgSJBx0X2QinboEcCPaMNK8Rq3 dVhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=AVSDB95rilSPFCbqIS7Vwa2mE5UN+mtkV1Ti5BvLwd0=; b=BxfHlszUvahkqyWSz9p4Fq/82lNpWSsJLt/fmbOA2ZsSEHEm3nQkBOrYDfkroBnZM/ ibS8YPFxQVUtsnxsRdLQaiCBvYbWa2ScJn0PCoOZYPfzggWu60RqHgGKpabo8R0HwEkO 5C14RhUSe466HsCfQRhfy+LX7/imrHcnrvcRtvRh08C8kmv26xGtVwrptEPuz4LDcnEG 1aDSkBP4NvwZhJi8G4t9i+D7h+M8xg/hgSgj0PXKxszLV4Soam7oE1N6GnbWPsAtOhbf yJulNYKeEYxLnOBripyJuIQpkPp/4iBFHC3ZWVkSgAbGjx1DwiamOo1/73h84VvOWL45 xZBA== X-Gm-Message-State: AOAM530+iXO74REJ3w2+h87OzplIoxzewiPl4qFK83Xa2/3/2svwvMzP mDfBqs79Hr4p41jlVhI2r+xt25ivnGU1v6uCuDCiqmjm X-Google-Smtp-Source: ABdhPJzE1DsWmneFjJKwyZfOQdmN8VxJRgEXPixg+gufKtABlK+gSSjg1wNRCDVhlA8yvusKCM166XruEzKM+eroNdU= X-Received: by 2002:a05:6512:1114:: with SMTP id l20mr8955330lfg.550.1631897089684; Fri, 17 Sep 2021 09:44:49 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a05:651c:2115:0:0:0:0 with HTTP; Fri, 17 Sep 2021 09:44:48 -0700 (PDT) In-Reply-To: References: <202109130411.18D4BUs3004805@gitrepo.freebsd.org> From: Mateusz Guzik Date: Fri, 17 Sep 2021 18:44:48 +0200 Message-ID: Subject: Re: git: 6fa041d7f125 - main - Measure latency of PMC interruptions To: Wojciech Macek Cc: Mitchell Horne , Wojciech Macek , src-committers , dev-commits-src-all@freebsd.org, dev-commits-src-main@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4HB0DM1vNNz4Sbf X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20210112 header.b=EsY4r5it; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of mjguzik@gmail.com designates 2a00:1450:4864:20::12d as permitted sender) smtp.mailfrom=mjguzik@gmail.com X-Spamd-Result: default: False [-2.01 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20210112]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[text/plain]; NEURAL_SPAM_SHORT(0.99)[0.993]; NEURAL_HAM_LONG(-1.00)[-1.000]; RCPT_COUNT_FIVE(0.00)[6]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MID_RHS_MATCH_FROMTLD(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::12d:from]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[dev-commits-src-all,dev-commits-src-main]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] X-BeenThere: dev-commits-src-all@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commit messages for all branches of the src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Sep 2021 16:44:52 -0000 ping? On 9/14/21, Wojciech Macek wrote: > Hi, > > Thanks for comments, I'll handle all of them as well as docs. > > Wojtek > > pon., 13 wrz 2021 o 22:06 Mitchell Horne napisa=C5= =82(a): > >> On Mon, Sep 13, 2021 at 1:11 AM Wojciech Macek wrote: >> > >> > The branch main has been updated by wma: >> > >> > URL: >> https://cgit.FreeBSD.org/src/commit/?id=3D6fa041d7f125db65f400af7f520a41= ff78d19cd7 >> > >> > commit 6fa041d7f125db65f400af7f520a41ff78d19cd7 >> > Author: Wojciech Macek >> > AuthorDate: 2021-09-13 04:08:32 +0000 >> > Commit: Wojciech Macek >> > CommitDate: 2021-09-13 04:08:32 +0000 >> > >> > Measure latency of PMC interruptions >> > >> > Add HWPMC events to measure latency. >> > Provide sysctl to choose the number of outstanding events which >> > trigger HWPMC event. >> > >> > Obtained from: Semihalf >> > Sponsored by: Stormshield >> > Differential revision: https://reviews.freebsd.org/D31283 >> > --- >> > sys/kern/kern_intr.c | 58 >> ++++++++++++++++++++++++++++++++++++++++++++++------ >> > 1 file changed, 52 insertions(+), 6 deletions(-) >> > >> > diff --git a/sys/kern/kern_intr.c b/sys/kern/kern_intr.c >> > index 1660414a50ef..19401877dfbd 100644 >> > --- a/sys/kern/kern_intr.c >> > +++ b/sys/kern/kern_intr.c >> > @@ -30,6 +30,7 @@ >> > __FBSDID("$FreeBSD$"); >> > >> > #include "opt_ddb.h" >> > +#include "opt_hwpmc_hooks.h" >> > #include "opt_kstack_usage_prof.h" >> > >> > #include >> > @@ -75,6 +76,7 @@ struct intr_thread { >> > struct thread *it_thread; /* Kernel thread. */ >> > int it_flags; /* (j) IT_* flags. */ >> > int it_need; /* Needs service. */ >> > + int it_waiting; /* Waiting in the runq. */ >> > }; >> > >> > /* Interrupt thread flags kept in it_flags */ >> > @@ -100,13 +102,19 @@ SYSCTL_INT(_hw, OID_AUTO, intr_storm_threshold, >> CTLFLAG_RWTUN, >> > static int intr_epoch_batch =3D 1000; >> > SYSCTL_INT(_hw, OID_AUTO, intr_epoch_batch, CTLFLAG_RWTUN, >> &intr_epoch_batch, >> > 0, "Maximum interrupt handler executions without re-entering >> epoch(9)"); >> > +#ifdef HWPMC_HOOKS >> > +static int intr_hwpmc_waiting_report_threshold =3D 1; >> > +SYSCTL_INT(_hw, OID_AUTO, intr_hwpmc_waiting_report_threshold, >> CTLFLAG_RWTUN, >> > + &intr_hwpmc_waiting_report_threshold, 1, >> > + "Threshold for reporting number of events in a workq"); >> > +#endif >> > static TAILQ_HEAD(, intr_event) event_list =3D >> > TAILQ_HEAD_INITIALIZER(event_list); >> > static struct mtx event_lock; >> > MTX_SYSINIT(intr_event_list, &event_lock, "intr event list", MTX_DEF)= ; >> > >> > static void intr_event_update(struct intr_event *ie); >> > -static int intr_event_schedule_thread(struct intr_event *ie); >> > +static int intr_event_schedule_thread(struct intr_event *ie, stru= ct >> trapframe *frame); >> > static struct intr_thread *ithread_create(const char *name); >> > static void ithread_destroy(struct intr_thread *ithread); >> > static void ithread_execute_handlers(struct proc *p, >> > @@ -115,6 +123,16 @@ static void ithread_loop(void *); >> > static void ithread_update(struct intr_thread *ithd); >> > static void start_softintr(void *); >> > >> > +#ifdef HWPMC_HOOKS >> > +#include >> > +PMC_SOFT_DEFINE( , , intr, all); >> > +PMC_SOFT_DEFINE( , , intr, ithread); >> > +PMC_SOFT_DEFINE( , , intr, filter); >> > +PMC_SOFT_DEFINE( , , intr, stray); >> > +PMC_SOFT_DEFINE( , , intr, schedule); >> > +PMC_SOFT_DEFINE( , , intr, waiting); >> > +#endif >> > + >> >> Hi Wojciech, >> >> Do you plan to document these new events in the pmc.soft(3) man page? >> >> Cheers, >> Mitchell >> >> >> > /* Map an interrupt type to an ithread priority. */ >> > u_char >> > intr_priority(enum intr_type flags) >> > @@ -773,7 +791,7 @@ intr_handler_barrier(struct intr_handler *handler) >> > } >> > if ((handler->ih_flags & IH_CHANGED) =3D=3D 0) { >> > handler->ih_flags |=3D IH_CHANGED; >> > - intr_event_schedule_thread(ie); >> > + intr_event_schedule_thread(ie, NULL); >> > } >> > while ((handler->ih_flags & IH_CHANGED) !=3D 0) >> > msleep(handler, &ie->ie_lock, 0, "ih_barr", 0); >> > @@ -872,7 +890,7 @@ intr_event_remove_handler(void *cookie) >> > KASSERT((handler->ih_flags & IH_DEAD) =3D=3D 0, >> > ("duplicate handle remove")); >> > handler->ih_flags |=3D IH_DEAD; >> > - intr_event_schedule_thread(ie); >> > + intr_event_schedule_thread(ie, NULL); >> > while (handler->ih_flags & IH_DEAD) >> > msleep(handler, &ie->ie_lock, 0, "iev_rmh", 0); >> > intr_event_update(ie); >> > @@ -944,7 +962,7 @@ intr_event_resume_handler(void *cookie) >> > } >> > >> > static int >> > -intr_event_schedule_thread(struct intr_event *ie) >> > +intr_event_schedule_thread(struct intr_event *ie, struct trapframe >> *frame) >> > { >> > struct intr_entropy entropy; >> > struct intr_thread *it; >> > @@ -986,11 +1004,28 @@ intr_event_schedule_thread(struct intr_event *i= e) >> > atomic_store_rel_int(&it->it_need, 1); >> > thread_lock(td); >> > if (TD_AWAITING_INTR(td)) { >> > +#ifdef HWPMC_HOOKS >> > + atomic_set_int(&it->it_waiting, 0); >> > + if (frame !=3D NULL) >> > + PMC_SOFT_CALL_TF( , , intr, schedule, frame); >> > + else >> > + PMC_SOFT_CALL( , , intr, schedule); >> > +#endif >> > CTR3(KTR_INTR, "%s: schedule pid %d (%s)", __func__, >> td->td_proc->p_pid, >> > td->td_name); >> > TD_CLR_IWAIT(td); >> > sched_add(td, SRQ_INTR); >> > } else { >> > +#ifdef HWPMC_HOOKS >> > + atomic_add_int(&it->it_waiting, 1); >> > + >> > + if (atomic_load_int(&it->it_waiting) >=3D >> intr_hwpmc_waiting_report_threshold) { >> > + if (frame !=3D NULL) >> > + PMC_SOFT_CALL_TF( , , intr, waiting, >> frame); >> > + else >> > + PMC_SOFT_CALL( , , intr, waiting); >> > + } >> > +#endif >> > CTR5(KTR_INTR, "%s: pid %d (%s): it_need %d, state %d"= , >> > __func__, td->td_proc->p_pid, td->td_name, >> it->it_need, TD_GET_STATE(td)); >> > thread_unlock(td); >> > @@ -1083,7 +1118,7 @@ swi_sched(void *cookie, int flags) >> > #endif >> > } else { >> > VM_CNT_INC(v_soft); >> > - error =3D intr_event_schedule_thread(ie); >> > + error =3D intr_event_schedule_thread(ie, NULL); >> > KASSERT(error =3D=3D 0, ("stray software interrupt")); >> > } >> > } >> > @@ -1374,12 +1409,23 @@ intr_event_handle(struct intr_event *ie, struc= t >> trapframe *frame) >> > ret =3D ih->ih_filter(frame); >> > else >> > ret =3D ih->ih_filter(ih->ih_argument); >> > +#ifdef HWPMC_HOOKS >> > + PMC_SOFT_CALL_TF( , , intr, all, frame); >> > +#endif >> > KASSERT(ret =3D=3D FILTER_STRAY || >> > ((ret & (FILTER_SCHEDULE_THREAD | FILTER_HANDLED)) >> !=3D 0 && >> > (ret & ~(FILTER_SCHEDULE_THREAD | FILTER_HANDLED)) >> =3D=3D 0), >> > ("%s: incorrect return value %#x from %s", __func_= _, >> ret, >> > ih->ih_name)); >> > filter =3D filter || ret =3D=3D FILTER_HANDLED; >> > +#ifdef HWPMC_HOOKS >> > + if (ret & FILTER_SCHEDULE_THREAD) >> > + PMC_SOFT_CALL_TF( , , intr, ithread, frame); >> > + else if (ret & FILTER_HANDLED) >> > + PMC_SOFT_CALL_TF( , , intr, filter, frame); >> > + else if (ret =3D=3D FILTER_STRAY) >> > + PMC_SOFT_CALL_TF( , , intr, stray, frame); >> > +#endif >> > >> > /* >> > * Wrapper handler special handling: >> > @@ -1416,7 +1462,7 @@ intr_event_handle(struct intr_event *ie, struct >> trapframe *frame) >> > if (thread) { >> > int error __unused; >> > >> > - error =3D intr_event_schedule_thread(ie); >> > + error =3D intr_event_schedule_thread(ie, frame); >> > KASSERT(error =3D=3D 0, ("bad stray interrupt")); >> > } >> > critical_exit(); >> > _______________________________________________ > dev-commits-src-all@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/dev-commits-src-all > To unsubscribe, send any mail to > "dev-commits-src-all-unsubscribe@freebsd.org" > --=20 Mateusz Guzik