From owner-dev-commits-src-all@freebsd.org Mon May 3 18:12:59 2021 Return-Path: Delivered-To: dev-commits-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2725563980E for ; Mon, 3 May 2021 18:12:59 +0000 (UTC) (envelope-from kevin.bowling@kev009.com) Received: from mail-yb1-xb2f.google.com (mail-yb1-xb2f.google.com [IPv6:2607:f8b0:4864:20::b2f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FYrgH0S4Gz3L78 for ; Mon, 3 May 2021 18:12:58 +0000 (UTC) (envelope-from kevin.bowling@kev009.com) Received: by mail-yb1-xb2f.google.com with SMTP id x131so8707202ybg.11 for ; Mon, 03 May 2021 11:12:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kev009.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=E+BxIIkbEP5jcScv3YzphzWCFE1GMuhz8eC6TiUgJWk=; b=GGiWIireaKVRQFYiWDPgjNUslxMtx45KtzkEQcQtQ/5SoDPO2bTGQiVmVSMmBcE6v9 4Z2DOBlbTqy+VblDZgLvVTq9d9Wg6Ym3/VKAThqoAJbPMvRvCuCqfOU/2zAHuiJIM7n7 uT9UVokevoLztzwJtJaGhKQFmt4IjG5LWGtfw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=E+BxIIkbEP5jcScv3YzphzWCFE1GMuhz8eC6TiUgJWk=; b=McoBMiDOcO+ARhQIH4Trs5a59j+kBQ83nKQ4W05nOm6WFf8LOlADXmUb51cyHNPoOa gI9r3O19Tre9yiDUI+KI7Z/YpO4H1zRHPLlNjDX5c2uWNHRVUWNHvara9ue0j40qwICc hblZ6zy5wH/RoVcxaRnM2B8rOHLIX82VHoJyaAwmg23jy1unW9hI/rou6LoYoQ7QsKin 7QvgJ8P7YCxXh7VmKJAUWJIURyR56b/9xOnHp1sgjp77nhQBv93dg3BLpO09aRe1WpxH dPsyIHBXkvOqUyyJjcRHyZIo5apBS3pJe0jL/zA4Az0guTqmdVOYGNRq705EEewsAAfm DMdA== X-Gm-Message-State: AOAM531XfiJ5Fq794zN+aNCd+yVIirJHf8iIuA3ZRaYRzkN1gRAu5J1u SNQQrMJRNGndIpaTOMHT4v2uQXwJAN2BY54aTv+kjg== X-Google-Smtp-Source: ABdhPJxjwlUd9/5ML2za63P3+4GVWCcWKqt0AF9hy1qOFXC70Ieuz1daxT7sacDEatbF5R90n7mekYsINvont0kft5Y= X-Received: by 2002:a25:ce51:: with SMTP id x78mr19273112ybe.87.1620065577873; Mon, 03 May 2021 11:12:57 -0700 (PDT) MIME-Version: 1.0 References: <202105031755.143Htj33046938@gitrepo.freebsd.org> In-Reply-To: <202105031755.143Htj33046938@gitrepo.freebsd.org> From: Kevin Bowling Date: Mon, 3 May 2021 11:12:46 -0700 Message-ID: Subject: Re: git: 64881da47807 - main - ixgbe: Restore AIM support To: Mark Johnston Cc: dev-commits-src-all@freebsd.org, dev-commits-src-main@freebsd.org, src-committers@freebsd.org X-Rspamd-Queue-Id: 4FYrgH0S4Gz3L78 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: dev-commits-src-all@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commit messages for all branches of the src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 May 2021 18:12:59 -0000 Should we enable this by default? My primary concern in the review is not making things like this require expert tuning. If it is expected to be necessary, which Gallatin@ makes a case for, let=E2=80=99s toggle it on by = default. On Mon, May 3, 2021 at 10:56 AM Mark Johnston wrote: > The branch main has been updated by markj: > > URL: > https://cgit.FreeBSD.org/src/commit/?id=3D64881da478071431a2d9e62613997a5= 772c56cdf > > commit 64881da478071431a2d9e62613997a5772c56cdf > Author: Sai Rajesh Tallamraju > AuthorDate: 2021-05-03 17:45:00 +0000 > Commit: Mark Johnston > CommitDate: 2021-05-03 17:47:14 +0000 > > ixgbe: Restore AIM support > > AIM (adaptive interrupt moderation) was part of BSD11 driver. Upon > IFLIB > migration, AIM feature got lost. Re-introducing AIM back into IFLIB > based IXGBE driver. > > One caveat is that in BSD11 driver, a queue comprises both Rx and Tx > ring. Starting from BSD12, Rx and Tx have their own queues and rings. > Also, IRQ is now only configured for Rx side. So, when AIM is > re-enabled, we should now consider only Rx stats for configuring EITR > register in contrast to BSD11 where Rx and Tx stats were considered t= o > manipulate EITR register. > > Reviewed by: gallatin, markj > Sponsored by: NetApp, Inc. > MFC after: 2 weeks > Differential Revision: https://reviews.freebsd.org/D27344 > --- > sys/dev/ixgbe/if_ix.c | 73 > +++++++++++++++++++++++++++++++++++++++++++++++++++ > sys/dev/ixgbe/ixgbe.h | 1 + > 2 files changed, 74 insertions(+) > > diff --git a/sys/dev/ixgbe/if_ix.c b/sys/dev/ixgbe/if_ix.c > index 489e891cf509..fbe4f1df38ca 100644 > --- a/sys/dev/ixgbe/if_ix.c > +++ b/sys/dev/ixgbe/if_ix.c > @@ -347,6 +347,16 @@ static int ixgbe_enable_rss =3D 1; > SYSCTL_INT(_hw_ix, OID_AUTO, enable_rss, CTLFLAG_RDTUN, > &ixgbe_enable_rss, 0, > "Enable Receive-Side Scaling (RSS)"); > > +/* > + * AIM: Adaptive Interrupt Moderation > + * which means that the interrupt rate > + * is varied over time based on the > + * traffic for that interrupt vector > + */ > +static int ixgbe_enable_aim =3D FALSE; > +SYSCTL_INT(_hw_ix, OID_AUTO, enable_aim, CTLFLAG_RWTUN, > &ixgbe_enable_aim, 0, > + "Enable adaptive interrupt moderation"); > + > #if 0 > /* Keep running tab on them for sanity check */ > static int ixgbe_total_ports; > @@ -2100,6 +2110,60 @@ fail: > return (error); > } /* ixgbe_if_msix_intr_assign */ > > +static inline void > +ixgbe_perform_aim(struct adapter *adapter, struct ix_rx_queue *que) > +{ > + uint32_t newitr =3D 0; > + struct rx_ring *rxr =3D &que->rxr; > + > + /* > + * Do Adaptive Interrupt Moderation: > + * - Write out last calculated setting > + * - Calculate based on average size over > + * the last interval. > + */ > + if (que->eitr_setting) { > + IXGBE_WRITE_REG(&adapter->hw, IXGBE_EITR(que->msix), > + que->eitr_setting); > + } > + > + que->eitr_setting =3D 0; > + /* Idle, do nothing */ > + if (rxr->bytes =3D=3D 0) { > + return; > + } > + > + if ((rxr->bytes) && (rxr->packets)) { > + newitr =3D (rxr->bytes / rxr->packets); > + } > + > + newitr +=3D 24; /* account for hardware frame, crc */ > + /* set an upper boundary */ > + newitr =3D min(newitr, 3000); > + > + /* Be nice to the mid range */ > + if ((newitr > 300) && (newitr < 1200)) { > + newitr =3D (newitr / 3); > + } else { > + newitr =3D (newitr / 2); > + } > + > + if (adapter->hw.mac.type =3D=3D ixgbe_mac_82598EB) { > + newitr |=3D newitr << 16; > + } else { > + newitr |=3D IXGBE_EITR_CNT_WDIS; > + } > + > + /* save for next interrupt */ > + que->eitr_setting =3D newitr; > + > + /* Reset state */ > + rxr->bytes =3D 0; > + rxr->packets =3D 0; > + > + return; > +} > + > /********************************************************************* > * ixgbe_msix_que - MSI-X Queue Interrupt Service routine > **********************************************************************/ > @@ -2117,6 +2181,11 @@ ixgbe_msix_que(void *arg) > ixgbe_disable_queue(adapter, que->msix); > ++que->irqs; > > + /* Check for AIM */ > + if (adapter->enable_aim) { > + ixgbe_perform_aim(adapter, que); > + } > + > return (FILTER_SCHEDULE_THREAD); > } /* ixgbe_msix_que */ > > @@ -2575,6 +2644,10 @@ ixgbe_add_device_sysctls(if_ctx_t ctx) > adapter, 0, ixgbe_sysctl_advertise, "I", > IXGBE_SYSCTL_DESC_ADV_SPEED); > > + adapter->enable_aim =3D ixgbe_enable_aim; > + SYSCTL_ADD_INT(ctx_list, child, OID_AUTO, "enable_aim", CTLFLAG_R= W, > + &adapter->enable_aim, 0, "Interrupt Moderation"); > + > #ifdef IXGBE_DEBUG > /* testing sysctls (for all devices) */ > SYSCTL_ADD_PROC(ctx_list, child, OID_AUTO, "power_state", > diff --git a/sys/dev/ixgbe/ixgbe.h b/sys/dev/ixgbe/ixgbe.h > index 30dd1d5368fb..31d5cc41c066 100644 > --- a/sys/dev/ixgbe/ixgbe.h > +++ b/sys/dev/ixgbe/ixgbe.h > @@ -408,6 +408,7 @@ struct adapter { > > /* Info about the interface */ > int advertise; /* link speeds */ > + int enable_aim; /* adaptive interrupt > moderation */ > bool link_active; > u16 num_segs; > u32 link_speed; > _______________________________________________ > dev-commits-src-main@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/dev-commits-src-main > To unsubscribe, send any mail to " > dev-commits-src-main-unsubscribe@freebsd.org" >