From nobody Mon Jul 22 03:57:27 2024 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WS61M4wsSz5RDY4; Mon, 22 Jul 2024 03:57:27 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WS61M2dThz41JY; Mon, 22 Jul 2024 03:57:27 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1721620647; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=CpFAHjFInBQ2N0Kc27GhxCtYHwk9ZQfJ5Ji5OUx7hhc=; b=H8jNq81nMjp2JzcOdI+u2BuJgdYnf6+HVha1OOr2feEKoroHAIEOpXguxTLMOnFGSeKZMp tHIj69t5umgoWxMVwY8yxZyhB3W40jPgKoEFJHOax5kcOXP3unFZDG2p2ym95u7QBfkz1I QUZbM/Ee/37sc0QEJ6ybe9RP9Iej127+iD+FrdwiIYNRvH2lKk9zu8buVlrMSefuGTVnLy YCdOfihcMnZQz5f1ALucj1KGA9Fpd7G4TKvCtV8p8Bk6wZtC64AoQ+5VlLE87MlMgMcIQb dd3yCs6ZkKQwUym+d38dTGrm88/Ws3YblGg5cuDzZcExmDlkLVzu/J3gGjcnog== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1721620647; a=rsa-sha256; cv=none; b=YnzocOGCMcfKPhb6yfWUFThL7Y+mgysHbRroFH1x8hMxrCsAD0Rlrz1/HaUzljMs5RVjF3 6kn/zfCvssI1UrcbJbxm/3kU/G7UGf1S02+WHWs+Vbw3U0dtNMKIju8ZaTweb76I46vexZ pJiQiRdnJ9NEsiPnTViu/1BFipnCDQvYfO4HLGjhGsSnmOOz5DB4cLYzdb5p7KVB/C71Ac leHz+7dltqRVlvHQhXasMPhCYUJ7JXT8iVYI18XkWK+2MP03IctK7H/7nE5+5G/HBVcZrG 4vMk/rNoC3T9SHqSNgvr0IKxJ4q9lkV4VaWnV3OhRhUi9rGjU99FjXFnwTOZUA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1721620647; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=CpFAHjFInBQ2N0Kc27GhxCtYHwk9ZQfJ5Ji5OUx7hhc=; b=SDEkEgcFiRu7kx8sR43Ovlvi637SSDJtgifYWIYxx3yn2+5VCjxsZttmVEeuQCWDAw0Y6H 41fBx7AaU2CKlEVIk347BjMVq1Mss7THAWTGWagGm40EhdduuN86KWQwnR+P6fi7PuGbkm kPDtwlBDx7WnizpB3v8MK0TIAeenFB36vvp96VKPKBbvplAWonl0XY4ethFsxhSthhZ7Rf 7kxUbVDP7Sqa2wDZcL+sfC0jmYGh3eienyDjjIMZMcAREhPQuAbXnh2Tmgz64Y4SiFocIY 14JTd4MZWL07vH9G8s+itmFrTqfm4U7RnMjCCUYpQrhrFT6Pcb95YzIDvbDDaw== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4WS61M2DgLzSvk; Mon, 22 Jul 2024 03:57:27 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 46M3vRH8014933; Mon, 22 Jul 2024 03:57:27 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 46M3vRxa014930; Mon, 22 Jul 2024 03:57:27 GMT (envelope-from git) Date: Mon, 22 Jul 2024 03:57:27 GMT Message-Id: <202407220357.46M3vRxa014930@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Navdeep Parhar Subject: git: fd3aca5b4196 - main - cxgbe(4): Stop and restart the atid allocator with the LLD. List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: np X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: fd3aca5b41968421f243b760ac5733f39f25cc56 Auto-Submitted: auto-generated The branch main has been updated by np: URL: https://cgit.FreeBSD.org/src/commit/?id=fd3aca5b41968421f243b760ac5733f39f25cc56 commit fd3aca5b41968421f243b760ac5733f39f25cc56 Author: Navdeep Parhar AuthorDate: 2024-07-12 23:53:22 +0000 Commit: Navdeep Parhar CommitDate: 2024-07-22 03:26:42 +0000 cxgbe(4): Stop and restart the atid allocator with the LLD. atids are used by both filters and TOE and the atid table is in the base driver (LLD). New atids cannot be allocated when the allocator is stopped but existing ones can still be freed. It is expected that the owners of outstanding atids will release them in their own stop processing, before the adapter is restarted. MFC after: 1 month Sponsored by: Chelsio Communications --- sys/dev/cxgbe/offload.h | 1 + sys/dev/cxgbe/t4_main.c | 33 ++++++++++++++++++++++++++++++++- 2 files changed, 33 insertions(+), 1 deletion(-) diff --git a/sys/dev/cxgbe/offload.h b/sys/dev/cxgbe/offload.h index 877fdb6b0a89..b17056dc0338 100644 --- a/sys/dev/cxgbe/offload.h +++ b/sys/dev/cxgbe/offload.h @@ -151,6 +151,7 @@ struct tid_info { union aopen_entry *atid_tab; union aopen_entry *afree; u_int atids_in_use; + bool atid_alloc_stopped; /* High priority filters and normal filters share the lock and cv. */ struct mtx ftid_lock __aligned(CACHE_LINE_SIZE); diff --git a/sys/dev/cxgbe/t4_main.c b/sys/dev/cxgbe/t4_main.c index 1fb560124fb0..f1f4b2d26fdf 100644 --- a/sys/dev/cxgbe/t4_main.c +++ b/sys/dev/cxgbe/t4_main.c @@ -780,6 +780,8 @@ static int t4_alloc_irq(struct adapter *, struct irq *, int rid, static int t4_free_irq(struct adapter *, struct irq *); static void t4_init_atid_table(struct adapter *); static void t4_free_atid_table(struct adapter *); +static void stop_atid_allocator(struct adapter *); +static void restart_atid_allocator(struct adapter *); static void get_regs(struct adapter *, struct t4_regdump *, uint8_t *); static void vi_refresh_stats(struct vi_info *); static void cxgbe_refresh_stats(struct vi_info *); @@ -2097,6 +2099,9 @@ stop_lld(struct adapter *sc) } end_synchronized_op(sc, 0); + + stop_atid_allocator(sc); + return (rc); } @@ -2447,6 +2452,9 @@ restart_lld(struct adapter *sc) done: end_synchronized_op(sc, 0); free(old_state, M_CXGBE); + + restart_atid_allocator(sc); + return (rc); } @@ -3941,6 +3949,7 @@ t4_init_atid_table(struct adapter *sc) mtx_init(&t->atid_lock, "atid lock", NULL, MTX_DEF); t->afree = t->atid_tab; t->atids_in_use = 0; + t->atid_alloc_stopped = false; for (i = 1; i < t->natids; i++) t->atid_tab[i - 1].next = &t->atid_tab[i]; t->atid_tab[t->natids - 1].next = NULL; @@ -3962,6 +3971,28 @@ t4_free_atid_table(struct adapter *sc) t->atid_tab = NULL; } +static void +stop_atid_allocator(struct adapter *sc) +{ + struct tid_info *t = &sc->tids; + + mtx_lock(&t->atid_lock); + t->atid_alloc_stopped = true; + mtx_unlock(&t->atid_lock); +} + +static void +restart_atid_allocator(struct adapter *sc) +{ + struct tid_info *t = &sc->tids; + + mtx_lock(&t->atid_lock); + KASSERT(t->atids_in_use == 0, + ("%s: %d atids still in use.", __func__, t->atids_in_use)); + t->atid_alloc_stopped = false; + mtx_unlock(&t->atid_lock); +} + int alloc_atid(struct adapter *sc, void *ctx) { @@ -3969,7 +4000,7 @@ alloc_atid(struct adapter *sc, void *ctx) int atid = -1; mtx_lock(&t->atid_lock); - if (t->afree) { + if (t->afree && !t->atid_alloc_stopped) { union aopen_entry *p = t->afree; atid = p - t->atid_tab;