From nobody Sun Aug 9 19:47:37 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4hJ7hb0pNZz6p0JR for ; Sun, 09 Aug 2026 19:47:43 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "YR1" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4hJ7hZ6Ns3z4KKr for ; Sun, 09 Aug 2026 19:47:42 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1786304862; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=863AYhxvl14jeYVI6rqJ1l+0ZT3ukNXTvpALukGjaDM=; b=lx5LzX5Lmye9vsYCXC9YYfJCXtOs5hOfGykZa8Ly8dFXwPSYjnt7LBrmNDJEwuZWqQxB5V FZoIzr3hIkkxvos51GzS/ZdKAY5hUfXJzumhtypgn4QVFOzDzCHS1l2229qMiTwNyUST0R L60lU/sTOuh8whfDxdfzxmZR1dA3n2dWFXNxFNawMTcJBywwX/DCp2Hdx8AWGY/zAzZMEB gzjWFuODnt61NQxsDroFQ1gq1T56o0leB0WMB8OEAZSwYdZoxEd9R3J+OEl0a5BgVjNDnz NiEZ9F6tMal9Ci+EIvoGDEtWe7hYX3iCyFYfEUhlz0c/n7vEoBHfFPSMkvOxhw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1786304862; a=rsa-sha256; cv=none; b=DKY5HKeIjkNNnmzmtickfpm2h2CMYx5D5O1NGh+w4+gTVsNcJa8tFBA+daHMKLXBMe5/7q kffT8W2/Fbw37vgz2aFuT4PKqr0x2rS/G/KoKdc8k1axx83YDUxdkYof36atCbEBE/2Bbw iS3Atzy0bHgkI2+Z19Nb2WcYL83Mv5fHKGLQUUIpAU+RKzqNgIjUACmgHOYdM+fAEshPwD pGelEVF8idb91cMkkAQUi/7M9c9e3jSD4ZxyZ1t4M8hkMmWSADeoiWH3NtCoDii1rYXpc/ lbnmUpZRmY7sjmQxj49xzfYT+rU3HiesubxbVAcn/GWstpabjjfw2TgcEflouA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1786304862; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=863AYhxvl14jeYVI6rqJ1l+0ZT3ukNXTvpALukGjaDM=; b=GSIfx++Aa0tQ731GbEpRQZox2qa075bDiW2Fe49nkxqAUuEeu0WMLPUTIneRAuREFBZgS6 bUzg8D8pMwSYpvshR/KSAzjh7Jzl8k9/06MbUA28f2W1L/Uh5D0CTpBLVTJbJyUOdVqwl/ pQZQ8xXnfgeiI9PIJnBwRkgqxTPBPPNcuPzEwkgHlW6oHsxRmy/H8UFROGUKynHNuq8CtF sJBzYidlM5IqkfdCAGSAh9eUpRZOvwdTDO2S+YgMh1Xzu9rH9kPPpgazb8BMhZ0vWtY59v QoL+yfOjlxNn54QUcZnE5vnN0sGuMilv2Pk5xZ+Ec3ZYn7pA1ZdP15/gNSCYlQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4hJ7hZ2ZDDzdXX for ; Sun, 09 Aug 2026 19:47:42 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 1bd08 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Sun, 09 Aug 2026 19:47:37 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Nick Price Subject: git: af3f2dd124f0 - main - aq(4): report link transitions and previously silent failures List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: nprice X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: af3f2dd124f07342c273dc017bc5e73639e446a5 Auto-Submitted: auto-generated Date: Sun, 09 Aug 2026 19:47:37 +0000 Message-Id: <6a78d959.1bd08.6d85d4b7@gitrepo.freebsd.org> The branch main has been updated by nprice: URL: https://cgit.FreeBSD.org/src/commit/?id=af3f2dd124f07342c273dc017bc5e73639e446a5 commit af3f2dd124f07342c273dc017bc5e73639e446a5 Author: Nick Price AuthorDate: 2026-08-09 19:47:31 +0000 Commit: Nick Price CommitDate: 2026-08-09 19:47:31 +0000 aq(4): report link transitions and previously silent failures A link flap left nothing in the log to work from. Both the link up and link down messages were gated on bootverbose while the message for a speed change that keeps carrier was not, so a default kernel was silent about a flap yet loud about a downshift -- the inverse of what an operator wants. The generic message from if_link_state_change() carries no speed, so gating the driver's own left the negotiated rate unrecorded. Report both transitions unconditionally. Say more than the rate. aq_hw_get_link_state() already negotiates flow control and throws it away, and Atlantic 2 reports duplex and EEE in the same link status word the rate comes from; decode them through a new get_link_info firmware op and name all of it on the up transition. EEE matters for a flap: low power idle transitions are a common source of marginal link trouble on multi-gigabit copper, and whether it was active is otherwise invisible. Give the down transition a cause. The PHY global fault code was only consulted from the thermal state machine, so an ordinary link loss reported nothing at all. Read the fault code, the firmware link state and the PHY temperature once per transition and append whatever is available. The firmware raises a fault one poll after it drops the link, so a thermal trip usually shows only its temperature here and aq_thermal_poll() names it on the following poll; the temperature alone is enough to separate a hot PHY from a cable event. Warn before the PHY trips rather than only after. The Atlantic 2 health monitor word carries a hot warning bit next to the ready and fault bits that nothing decoded. Report both edges of it from the thermal poll, so an adapter that is approaching its shutdown threshold says so while the link is still up. Expose the firmware's own link transition counters. The Atlantic 2 A0 statistics layout opens with link_up and link_down, which were read out of the firmware on every statistics poll and discarded. Publish them as dev.aq.N.fw_link_up and fw_link_down so a single flap can be told from a link that has been flapping all night. The B0 layout has no equivalent, so the op reports ENOTSUP there and the nodes are not created, matching how the temperature node is handled. Stop announcing a link state that was never read. The return value of aq_hw_get_link_state() was discarded, so a failed read would have been announced as link down. No firmware backend can fail that call today -- all three decode a register with no error path -- but the caller no longer depends on that, and it says so once if it ever starts failing. Report the hardware failures that were being discarded. The driver already reports the errors it keeps, so what stayed quiet was the set of calls whose result was never examined at all. None of these are expected to fail, which is precisely why a failure needs to say so: each one leaves the interface running but misconfigured in a way that presents as a network problem rather than a driver problem. aq_if_init() discarded aq_hw_start(), aq_hw_rss_hash_set(), aq_hw_rss_set() and aq_hw_udp_rss_enable(), so a datapath that never started or an indirection table that was never programmed showed up only as an interface that passes no traffic or delivers every flow to one queue. aq_mc_filter_apply() discarded aq_hw_mac_addr_set(), so a multicast address the stack believes is programmed could silently not be; report the address that failed and leave the filter slot for the next one instead of burning it. aq_update_vlan_filters() reported only the last of its three register writes. aq_if_stop() discarded both ring stop calls and the MAC reset, and a MAC that did not reset can still be mastering the bus. aq_if_detach() and aq_if_suspend() discarded aq_hw_deinit(). The interrupt moderation update on a link speed change was dropped as well; it runs only on a transition, so reporting it cannot become noisy. aq_if_attach_pre() discarded aq_hw_capabilities(), which is the only behavioral change here: it now fails the attach rather than continuing with an unset media type and an empty link speed mask, which would attach an interface that can never negotiate a link. It returns an error only for a device the probe table does not cover, so it is not reachable in practice. Document the resulting sysctls, along with the existing temperature and tracing nodes, which had no manual page coverage. Tested on an AQC113C (Atlantic 2 B0, firmware 1.5.38). Link up reports "speed=10000, full-duplex, flowcontrol none, EEE off", and "speed=1000" after a forced renegotiation, so the rate and duplex are read rather than assumed. A cable pull reports "link DOWN, F/W link state 0, temp 59 C" with the PHY fault clause correctly absent, which is what separates a cable event from a thermal trip. The B0 interface reports ENOTSUP for the link counters, so those two nodes are correctly not created. Traffic is unaffected: ten flows spread over all eight RX queues with no errors and no drops. Reviewed by: adrian Differential Revision: https://reviews.freebsd.org/D58749 Signed-off-by: Nick Price --- share/man/man4/aq.4 | 31 ++++++++++++- sys/dev/aq/aq2_fw.c | 66 ++++++++++++++++++++++++++ sys/dev/aq/aq2_hw.h | 1 + sys/dev/aq/aq_device.h | 2 + sys/dev/aq/aq_fw.h | 9 ++++ sys/dev/aq/aq_hw.h | 6 +++ sys/dev/aq/aq_irq.c | 123 +++++++++++++++++++++++++++++++++++++++++++++---- sys/dev/aq/aq_main.c | 97 ++++++++++++++++++++++++++++++++------ 8 files changed, 311 insertions(+), 24 deletions(-) diff --git a/share/man/man4/aq.4 b/share/man/man4/aq.4 index caf55de1083b..50b3fd853e8c 100644 --- a/share/man/man4/aq.4 +++ b/share/man/man4/aq.4 @@ -3,7 +3,7 @@ .\" .\" SPDX-License-Identifier: BSD-2-Clause .\" -.Dd July 9, 2026 +.Dd August 6, 2026 .Dt AQ 4 .Os .Sh NAME @@ -79,6 +79,35 @@ Marvell AQC115C (10Mb/100Mb/1Gb/2.5Gb RJ45) .It Marvell AQC116C (10Mb/100Mb/1Gb RJ45) .El +.Sh SYSCTL VARIABLES +The following read-only variables are available per interface as +.Va dev.aq.%d.%s , +where +.Ar %d +is the interface number. +Each is present only when the adapter's firmware supports it. +.Bl -tag -width indent +.It Va temperature +The PHY die temperature. +.It Va fw_link_up +The number of link up transitions counted by the firmware. +.It Va fw_link_down +The number of link down transitions counted by the firmware. +Comparing this against the transitions logged by the driver +distinguishes a single link event from a flapping link. +.El +.Pp +The following variables tune the driver's own tracing: +.Bl -tag -width indent +.It Va debug +Trace verbosity: 0 disables tracing, 3 reports errors, 4 adds warnings, +5 adds traces and 6 adds detail. +The default is 3. +.It Va debug_categories +Bit mask selecting the traced categories: init 1, config 2, tx 4, rx 8, +intr 16 and fw 32. +All categories are enabled by default. +.El .Sh SEE ALSO .Xr arp 4 , .Xr miibus 4 , diff --git a/sys/dev/aq/aq2_fw.c b/sys/dev/aq/aq2_fw.c index 9dc477dcf8e9..3c2ba7ad944c 100644 --- a/sys/dev/aq/aq2_fw.c +++ b/sys/dev/aq/aq2_fw.c @@ -39,10 +39,14 @@ static int aq2_fw_set_mode(struct aq_hw *hw, enum aq_hw_fw_mpi_state mode, enum aq_fw_link_speed speed); static int aq2_fw_get_mode(struct aq_hw *hw, enum aq_hw_fw_mpi_state *mode, enum aq_fw_link_speed *speed, enum aq_fw_link_fc *fc); +static int aq2_fw_get_link_info(struct aq_hw *hw, struct aq_hw_link_info *info); static int aq2_fw_get_mac_addr(struct aq_hw *hw, uint8_t *mac); static int aq2_fw_get_stats(struct aq_hw *hw, struct aq_hw_stats *stats); +static int aq2_fw_get_link_counters(struct aq_hw *hw, uint32_t *up, + uint32_t *down); static int aq2_fw_get_temp(struct aq_hw *hw, int *temp_mc); static int aq2_fw_get_thermal_limit(struct aq_hw *hw, int *limit_mc); +static int aq2_fw_get_phy_hot_warning(struct aq_hw *hw, bool *hot); /* Coherent OUT-window read, bracketed by the transaction id. */ static int @@ -369,6 +373,21 @@ aq2_fw_get_mode(struct aq_hw *hw, enum aq_hw_fw_mpi_state *modep, return (0); } +static int +aq2_fw_get_link_info(struct aq_hw *hw, struct aq_hw_link_info *info) +{ + uint32_t v; + + v = AQ_READ_REG(hw, AQ2_FW_INTERFACE_OUT_LINK_STATUS_REG); + + info->full_duplex = + (v & AQ2_FW_INTERFACE_OUT_LINK_STATUS_DUPLEX) != 0; + info->eee = (v & AQ2_FW_INTERFACE_OUT_LINK_STATUS_EEE) != 0; + info->state = v & AQ2_FW_INTERFACE_OUT_LINK_STATUS_STATE; + + return (0); +} + /* A2 firmware stats; layout depends on interface version. */ struct aq2_fw_statistics_a0 { uint32_t link_up; @@ -474,6 +493,30 @@ aq2_fw_get_stats(struct aq_hw *hw, struct aq_hw_stats *stats) return (0); } +/* Only the A0 interface layout carries the link transition counters. */ +static int +aq2_fw_get_link_counters(struct aq_hw *hw, uint32_t *up, uint32_t *down) +{ + union aq2_fw_statistics u; + int err; + + if (hw->aq2_iface != AQ2_FW_INTERFACE_OUT_VERSION_IFACE_VER_A0) + return (ENOTSUP); + + err = aq2_fw_interface_buffer_read(hw, AQ2_FW_INTERFACE_OUT_STATS_REG, + (uint32_t *)&u, sizeof(u)); + if (err != 0) { + trace_error(hw, dbg_fw, + "aq2> link counter read FAILED, error %d", err); + return (err); + } + + *up = u.a0.link_up; + *down = u.a0.link_down; + + return (0); +} + static int aq2_fw_get_temp(struct aq_hw *hw, int *temp_mc) { @@ -543,14 +586,37 @@ aq2_fw_get_phy_fault(struct aq_hw *hw, uint16_t *fault) return (0); } +static int +aq2_fw_get_phy_hot_warning(struct aq_hw *hw, bool *hot) +{ + uint32_t health; + int err; + + err = aq2_fw_interface_buffer_read(hw, + AQ2_FW_INTERFACE_OUT_PHY_HEALTH_MONITOR_REG, &health, + sizeof(health)); + if (err != 0) + return (err); + + if ((health & AQ2_FW_INTERFACE_OUT_PHY_HEALTH_MONITOR_READY) == 0) + return (ENXIO); + + *hot = (health & AQ2_FW_INTERFACE_OUT_PHY_HEALTH_MONITOR_HOT) != 0; + + return (0); +} + const struct aq_firmware_ops aq2_fw_ops = { .reset = aq2_fw_reset, .set_mode = aq2_fw_set_mode, .get_mode = aq2_fw_get_mode, + .get_link_info = aq2_fw_get_link_info, .get_mac_addr = aq2_fw_get_mac_addr, .get_stats = aq2_fw_get_stats, + .get_link_counters = aq2_fw_get_link_counters, .get_temp = aq2_fw_get_temp, .get_phy_fault = aq2_fw_get_phy_fault, + .get_phy_hot_warning = aq2_fw_get_phy_hot_warning, .phy_reset = NULL, /* A2 clears thermal shutdown on its own reset */ .thermal_arm = NULL, /* A2 firmware ships thermal shutdown armed */ .get_thermal_limit = aq2_fw_get_thermal_limit, diff --git a/sys/dev/aq/aq2_hw.h b/sys/dev/aq/aq2_hw.h index 1f68a9c4a1d0..c395af680433 100644 --- a/sys/dev/aq/aq2_hw.h +++ b/sys/dev/aq/aq2_hw.h @@ -237,6 +237,7 @@ aq_is_atlantic2(uint16_t device_id) #define AQ2_FW_INTERFACE_OUT_PHY_HEALTH_MONITOR_REG 0x13620 #define AQ2_FW_INTERFACE_OUT_PHY_HEALTH_MONITOR_READY 0x00000001 #define AQ2_FW_INTERFACE_OUT_PHY_HEALTH_MONITOR_FAULT 0x00000002 +#define AQ2_FW_INTERFACE_OUT_PHY_HEALTH_MONITOR_HOT 0x00000004 #define AQ2_FW_INTERFACE_OUT_PHY_TEMPERATURE 0x0000ff00 #define AQ2_FW_INTERFACE_OUT_PHY_TEMPERATURE_S 8 diff --git a/sys/dev/aq/aq_device.h b/sys/dev/aq/aq_device.h index 31c225bf2ddf..4f4c3335e4c8 100644 --- a/sys/dev/aq/aq_device.h +++ b/sys/dev/aq/aq_device.h @@ -123,6 +123,8 @@ struct aq_dev { bool linkup; uint32_t link_speed; /* Mbit/s last announced to the stack */ uint16_t phy_fault_last; /* last fault code reported */ + bool phy_hot_last; /* last over-temperature warning */ + bool link_read_failed; /* link state read is failing */ enum aq_thermal_state { AQ_THERMAL_NORMAL = 0, /* no thermal shutdown pending */ AQ_THERMAL_COOLING, /* shut down; waiting to cool */ diff --git a/sys/dev/aq/aq_fw.h b/sys/dev/aq/aq_fw.h index a7e880df51a7..bdac9f87cb08 100644 --- a/sys/dev/aq/aq_fw.h +++ b/sys/dev/aq/aq_fw.h @@ -65,15 +65,24 @@ struct aq_firmware_ops int (*set_mode)(struct aq_hw* hw, enum aq_hw_fw_mpi_state mode, enum aq_fw_link_speed speed); int (*get_mode)(struct aq_hw* hw, enum aq_hw_fw_mpi_state* mode, enum aq_fw_link_speed* speed, enum aq_fw_link_fc* fc); + /* Reports duplex, EEE and the firmware link state beyond the rate. */ + int (*get_link_info)(struct aq_hw* hw, struct aq_hw_link_info* info); + int (*get_mac_addr)(struct aq_hw* hw, uint8_t* mac_addr); int (*get_stats)(struct aq_hw* hw, struct aq_hw_stats* stats); + /* Reports the firmware's own link transition counters. */ + int (*get_link_counters)(struct aq_hw* hw, uint32_t* up, uint32_t* down); + /* Reports millidegrees Celsius. */ int (*get_temp)(struct aq_hw* hw, int* temp_mc); /* Reports the PHY global fault code; zero means no fault. */ int (*get_phy_fault)(struct aq_hw* hw, uint16_t* fault); + /* Reports the pre-shutdown PHY over-temperature warning. */ + int (*get_phy_hot_warning)(struct aq_hw* hw, bool* hot); + /* Resets the PHY (clears a latched thermal shutdown). */ int (*phy_reset)(struct aq_hw* hw); diff --git a/sys/dev/aq/aq_hw.h b/sys/dev/aq/aq_hw.h index 1156b33c1904..bf4925d139f9 100644 --- a/sys/dev/aq/aq_hw.h +++ b/sys/dev/aq/aq_hw.h @@ -162,6 +162,12 @@ struct aq_hw_fc_info { bool fc_tx; }; +struct aq_hw_link_info { + bool full_duplex; + bool eee; + uint8_t state; +}; + struct aq_hw { void *aq_dev; device_t dev; diff --git a/sys/dev/aq/aq_irq.c b/sys/dev/aq/aq_irq.c index 054492bfe3df..e52820ad77dc 100644 --- a/sys/dev/aq/aq_irq.c +++ b/sys/dev/aq/aq_irq.c @@ -39,6 +39,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include #include @@ -149,6 +150,34 @@ aq_thermal_report_shutdown(struct aq_dev *aq_dev) "holding link down until it cools\n"); } +/* Pre-trip warning: the PHY is hot but the firmware has not shut it down. */ +static void +aq_thermal_report_hot(struct aq_dev *aq_dev) +{ + struct aq_hw *hw = &aq_dev->hw; + int temp_mc; + bool hot; + + if (hw->fw_ops->get_phy_hot_warning == NULL || + hw->fw_ops->get_phy_hot_warning(hw, &hot) != 0) + return; + + /* Report each edge once; the bit stays set for the whole excursion. */ + if (hot == aq_dev->phy_hot_last) + return; + aq_dev->phy_hot_last = hot; + + if (!hot) { + device_printf(aq_dev->dev, "PHY temperature normal again\n"); + return; + } + if (hw->fw_ops->get_temp(hw, &temp_mc) == 0) + device_printf(aq_dev->dev, "PHY over-temperature warning, " + "temp %d C\n", temp_mc / 1000); + else + device_printf(aq_dev->dev, "PHY over-temperature warning\n"); +} + /* Recover after cooldown: A1 needs a PHY reset then re-init, A2 re-inits alone. */ static void aq_thermal_poll(struct aq_dev *aq_dev) @@ -159,6 +188,7 @@ aq_thermal_poll(struct aq_dev *aq_dev) switch (aq_dev->thermal_state) { case AQ_THERMAL_NORMAL: + aq_thermal_report_hot(aq_dev); if (aq_dev->linkup) return; /* The F/W raises the fault a poll after it drops the link. */ @@ -211,6 +241,72 @@ aq_thermal_poll(struct aq_dev *aq_dev) iflib_admin_intr_deferred(aq_dev->ctx); } +static const char * +aq_fc_string(const struct aq_hw_fc_info *fc) +{ + + if (fc->fc_rx && fc->fc_tx) + return ("rx/tx"); + if (fc->fc_rx) + return ("rx"); + if (fc->fc_tx) + return ("tx"); + return ("none"); +} + +static void +aq_link_report_up(struct aq_dev *aq_dev, uint32_t link_speed, + const struct aq_hw_fc_info *fc) +{ + struct aq_hw *hw = &aq_dev->hw; + struct aq_hw_link_info info; + + if (hw->fw_ops->get_link_info != NULL && + hw->fw_ops->get_link_info(hw, &info) == 0) { + device_printf(aq_dev->dev, "link UP: speed=%u, %s-duplex, " + "flowcontrol %s, EEE %s\n", link_speed, + info.full_duplex ? "full" : "half", aq_fc_string(fc), + info.eee ? "on" : "off"); + return; + } + device_printf(aq_dev->dev, "link UP: speed=%u, flowcontrol %s\n", + link_speed, aq_fc_string(fc)); +} + +/* + * The firmware raises a PHY fault a poll after it drops the link, so a + * thermal trip usually reports only its temperature here; aq_thermal_poll() + * names it on the following poll. + */ +static void +aq_link_report_down(struct aq_dev *aq_dev) +{ + struct aq_hw *hw = &aq_dev->hw; + struct aq_hw_link_info info; + struct sbuf sb; + char detail[96]; + int temp_mc; + uint16_t fault; + + sbuf_new(&sb, detail, sizeof(detail), SBUF_FIXEDLEN); + + if (hw->fw_ops->get_phy_fault != NULL && + hw->fw_ops->get_phy_fault(hw, &fault) == 0 && fault != 0) + sbuf_printf(&sb, ", PHY fault 0x%04x", fault); + if (hw->fw_ops->get_link_info != NULL && + hw->fw_ops->get_link_info(hw, &info) == 0) + sbuf_printf(&sb, ", F/W link state %u", info.state); + if (hw->fw_ops->get_temp != NULL && + hw->fw_ops->get_temp(hw, &temp_mc) == 0) + sbuf_printf(&sb, ", temp %d C", temp_mc / 1000); + + if (sbuf_finish(&sb) != 0) + device_printf(aq_dev->dev, "link DOWN\n"); + else + device_printf(aq_dev->dev, "link DOWN%s\n", sbuf_data(&sb)); + sbuf_delete(&sb); +} + void aq_if_update_admin_status(if_ctx_t ctx) { @@ -221,7 +317,17 @@ aq_if_update_admin_status(if_ctx_t ctx) struct aq_hw_fc_info fc_neg; - aq_hw_get_link_state(hw, &link_speed, &fc_neg); + int err; + + err = aq_hw_get_link_state(hw, &link_speed, &fc_neg); + if (err != 0 && !aq_dev->link_read_failed) { + aq_dev->link_read_failed = true; + device_printf(aq_dev->dev, "link state read failed, error %d; " + "holding the last known state\n", err); + } else if (err == 0 && aq_dev->link_read_failed) { + aq_dev->link_read_failed = false; + device_printf(aq_dev->dev, "link state read recovered\n"); + } /* A stopped or half-initialized interface has no link. */ running = (if_getdrvflags(iflib_get_ifp(ctx)) & IFF_DRV_RUNNING) != 0; @@ -229,12 +335,10 @@ aq_if_update_admin_status(if_ctx_t ctx) link_speed = 0; /* A retrain can change the speed without ever dropping the link. */ - if (link_speed != 0U && + if (err == 0 && link_speed != 0U && (!aq_dev->linkup || link_speed != aq_dev->link_speed)) { if (!aq_dev->linkup) { - if (bootverbose) - device_printf(aq_dev->dev, - "link UP: speed=%d\n", link_speed); + aq_link_report_up(aq_dev, link_speed, &fc_neg); aq_dev->phy_fault_last = 0; } else device_printf(aq_dev->dev, "link speed=%d\n", @@ -251,10 +355,11 @@ aq_if_update_admin_status(if_ctx_t ctx) aq_mediastatus_update(aq_dev, link_speed, &fc_neg); /* update ITR settings according new link speed */ - aq_hw_interrupt_moderation_set(hw); - } else if (link_speed == 0U && aq_dev->linkup) { /* link was UP */ - if (bootverbose) - device_printf(aq_dev->dev, "link DOWN\n"); + if (aq_hw_interrupt_moderation_set(hw) != 0) + device_printf(aq_dev->dev, + "could not update interrupt moderation\n"); + } else if (err == 0 && link_speed == 0U && aq_dev->linkup) { + aq_link_report_down(aq_dev); aq_dev->linkup = 0; aq_dev->link_speed = 0; diff --git a/sys/dev/aq/aq_main.c b/sys/dev/aq/aq_main.c index d958a2aa62c5..bde0b71cd458 100644 --- a/sys/dev/aq/aq_main.c +++ b/sys/dev/aq/aq_main.c @@ -405,7 +405,12 @@ aq_if_attach_pre(if_ctx_t ctx) __func__, rc); goto fail; } - aq_hw_capabilities(softc); + rc = aq_hw_capabilities(softc); + if (rc != 0) { + device_printf(softc->dev, "unsupported device %04x:%04x\n", + pci_get_vendor(softc->dev), pci_get_device(softc->dev)); + goto fail; + } rc = aq_hw_get_mac_permanent(hw, hw->mac_addr); if (rc != 0) { @@ -525,7 +530,8 @@ aq_if_detach(if_ctx_t ctx) sysctl_ctx_free(&softc->aq_sysctl_ctx); - aq_hw_deinit(&softc->hw); + if (aq_hw_deinit(&softc->hw) != 0) + device_printf(softc->dev, "could not shut the hardware down\n"); for (i = 0; i < softc->rx_rings_count; i++) iflib_irq_free(ctx, &softc->rx_rings[i]->irq); @@ -558,7 +564,9 @@ aq_if_suspend(if_ctx_t ctx) AQ_DBG_ENTER(); aq_if_stop(ctx); - aq_hw_deinit(&softc->hw); + if (aq_hw_deinit(&softc->hw) != 0) + device_printf(softc->dev, + "could not shut the hardware down for suspend\n"); /* iflib_device_suspend() does not stop the interface for us. */ if_setdrvflagbits(iflib_get_ifp(ctx), IFF_DRV_OACTIVE, IFF_DRV_RUNNING); @@ -810,15 +818,28 @@ aq_if_init(if_ctx_t ctx) aq_if_rx_queue_intr_enable(ctx, i); } - aq_hw_start(hw); + err = aq_hw_start(hw); + if (err != 0) + device_printf(softc->dev, "could not start the datapath: %d\n", + err); aq_if_enable_intr(ctx); - aq_hw_rss_hash_set(&softc->hw, softc->rss_key); - aq_hw_rss_set(&softc->hw, softc->rss_table); + err = aq_hw_rss_hash_set(&softc->hw, softc->rss_key); + if (err != 0) + device_printf(softc->dev, "could not set the RSS key: %d\n", + err); + err = aq_hw_rss_set(&softc->hw, softc->rss_table); + if (err != 0) + device_printf(softc->dev, + "could not set the RSS indirection table: %d\n", err); /* A2 selects UDP hashing per-protocol in REDIR2; A1 uses the filter. */ - if (!IS_CHIP_FEATURE(hw, ATLANTIC2)) - aq_hw_udp_rss_enable(hw, (aq_rss_hashconfig() & + if (!IS_CHIP_FEATURE(hw, ATLANTIC2)) { + err = aq_hw_udp_rss_enable(hw, (aq_rss_hashconfig() & (RSS_HASHTYPE_RSS_UDP_IPV4 | RSS_HASHTYPE_RSS_UDP_IPV6 | RSS_HASHTYPE_RSS_UDP_IPV6_EX)) != 0); + if (err != 0) + device_printf(softc->dev, + "could not configure UDP RSS hashing: %d\n", err); + } err = aq_hw_set_link_speed(hw, hw->link_rate); if (err != 0) device_printf(softc->dev, "could not set link speed: %d\n", err); @@ -848,15 +869,20 @@ aq_if_stop(if_ctx_t ctx) aq_if_disable_intr(ctx); for (i = 0; i < softc->tx_rings_count; i++) { - aq_ring_tx_stop(hw, softc->tx_rings[i]); + if (aq_ring_tx_stop(hw, softc->tx_rings[i]) != 0) + device_printf(softc->dev, + "could not stop TX ring %d\n", i); softc->tx_rings[i]->tx_head = 0; softc->tx_rings[i]->tx_tail = 0; } for (i = 0; i < softc->rx_rings_count; i++) { - aq_ring_rx_stop(hw, softc->rx_rings[i]); + if (aq_ring_rx_stop(hw, softc->rx_rings[i]) != 0) + device_printf(softc->dev, + "could not stop RX ring %d\n", i); } - aq_hw_reset(&softc->hw, true); + if (aq_hw_reset(&softc->hw, true) != 0) + device_printf(softc->dev, "could not reset the MAC on stop\n"); memset(&softc->last_stats, 0, sizeof(softc->last_stats)); /* Each bring-up gets its own budget of re-init attempts. */ softc->init_retries = 0; @@ -897,7 +923,11 @@ aq_mc_filter_apply(void *arg, struct sockaddr_dl *dl, u_int count) return (0); mac_addr = LLADDR(dl); - aq_hw_mac_addr_set(hw, mac_addr, count + 1); + if (aq_hw_mac_addr_set(hw, mac_addr, count + 1) != 0) { + device_printf(softc->dev, + "could not program multicast address %6D\n", mac_addr, ":"); + return (0); + } aq_log_detail(hw, "set %d mc address %6D", count + 1, mac_addr, ":"); return (1); @@ -1171,7 +1201,9 @@ aq_update_vlan_filters(struct aq_dev *softc) int vlan_tag = -1; int i; - hw_atl_b0_hw_vlan_promisc_set(hw, true); + if (hw_atl_b0_hw_vlan_promisc_set(hw, true) != 0) + device_printf(softc->dev, + "could not open the VLAN filter for update\n"); for (i = 0; i < AQ_HW_VLAN_MAX_FILTERS; i++) { bit_ffs_at(softc->vlan_tags, bit_pos, 4096, &vlan_tag); if (vlan_tag != -1) { @@ -1185,7 +1217,9 @@ aq_update_vlan_filters(struct aq_dev *softc) } } - hw_atl_b0_hw_vlan_set(hw, aq_vlans); + if (hw_atl_b0_hw_vlan_set(hw, aq_vlans) != 0) + device_printf(softc->dev, + "could not program the VLAN filter table\n"); if (hw_atl_b0_hw_vlan_promisc_set(hw, aq_is_vlan_promisc_required(softc) || (if_getflags(iflib_get_ifp(softc->ctx)) & IFF_PROMISC) != 0) != 0) @@ -1397,6 +1431,25 @@ aq_sysctl_temperature(SYSCTL_HANDLER_ARGS) return (sysctl_handle_int(oidp, &val, 0, req)); } +/* arg2 selects the counter: 0 = link up, 1 = link down. */ +static int +aq_sysctl_fw_link_counter(SYSCTL_HANDLER_ARGS) +{ + struct aq_dev *softc = arg1; + uint32_t up, down; + int error; + + if (softc->hw.fw_ops == NULL || + softc->hw.fw_ops->get_link_counters == NULL) + return (ENOTSUP); + + error = softc->hw.fw_ops->get_link_counters(&softc->hw, &up, &down); + if (error != 0) + return (error); + + return (sysctl_handle_32(oidp, arg2 == 0 ? &up : &down, 0, req)); +} + static void aq_add_stats_sysctls(struct aq_dev *softc) { @@ -1407,6 +1460,7 @@ aq_add_stats_sysctls(struct aq_dev *softc) struct aq_stats *stats = &softc->curr_stats; struct sysctl_oid *stat_node, *queue_node; struct sysctl_oid_list *stat_list, *queue_list; + uint32_t link_up, link_down; int temp_mc; #define QUEUE_NAME_LEN 32 @@ -1431,6 +1485,21 @@ aq_add_stats_sysctls(struct aq_dev *softc) CTLTYPE_INT | CTLFLAG_RD | CTLFLAG_MPSAFE, softc, 0, aq_sysctl_temperature, "IK", "PHY temperature"); + /* Only some firmware interface versions count link transitions. */ + if (softc->hw.fw_ops != NULL && + softc->hw.fw_ops->get_link_counters != NULL && + softc->hw.fw_ops->get_link_counters(&softc->hw, &link_up, + &link_down) != ENOTSUP) { + SYSCTL_ADD_PROC(ctx, child, OID_AUTO, "fw_link_up", + CTLTYPE_U32 | CTLFLAG_RD | CTLFLAG_MPSAFE, softc, 0, + aq_sysctl_fw_link_counter, "IU", + "Link up transitions counted by the firmware"); + SYSCTL_ADD_PROC(ctx, child, OID_AUTO, "fw_link_down", + CTLTYPE_U32 | CTLFLAG_RD | CTLFLAG_MPSAFE, softc, 1, + aq_sysctl_fw_link_counter, "IU", + "Link down transitions counted by the firmware"); + } + /* Driver Statistics */ for (int i = 0; i < softc->tx_rings_count; i++) { struct aq_ring *ring = softc->tx_rings[i];