From owner-svn-src-stable-12@freebsd.org Sun Jul 7 00:30:22 2019 Return-Path: Delivered-To: svn-src-stable-12@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0587B15D754C; Sun, 7 Jul 2019 00:30:22 +0000 (UTC) (envelope-from np@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8D7F988501; Sun, 7 Jul 2019 00:30:21 +0000 (UTC) (envelope-from np@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 675D913AF; Sun, 7 Jul 2019 00:30:21 +0000 (UTC) (envelope-from np@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x670ULlj001004; Sun, 7 Jul 2019 00:30:21 GMT (envelope-from np@FreeBSD.org) Received: (from np@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x670UKx9001002; Sun, 7 Jul 2019 00:30:20 GMT (envelope-from np@FreeBSD.org) Message-Id: <201907070030.x670UKx9001002@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: np set sender to np@FreeBSD.org using -f From: Navdeep Parhar Date: Sun, 7 Jul 2019 00:30:20 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-12@freebsd.org Subject: svn commit: r349795 - stable/12/sys/dev/cxgbe/tom X-SVN-Group: stable-12 X-SVN-Commit-Author: np X-SVN-Commit-Paths: stable/12/sys/dev/cxgbe/tom X-SVN-Commit-Revision: 349795 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 8D7F988501 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.96 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.96)[-0.957,0]; ASN(0.00)[asn:12874, ipnet:2000::/3, country:IT]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: svn-src-stable-12@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for only the 12-stable src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Jul 2019 00:30:22 -0000 Author: np Date: Sun Jul 7 00:30:20 2019 New Revision: 349795 URL: https://svnweb.freebsd.org/changeset/base/349795 Log: MFC r349242, r349501, r349514, and r349517. r349242: cxgbe/t4_tom: DDP_DEAD is a ddp flag and not a toepcb flag. The driver was in effect setting TPF_ABORT_SHUTDOWN on the toepcb instead of what was intended. Sponsored by: Chelsio Communications r349501: cxgbe/t4_tom: Mark the socket's receive as done before calling handle_ddp_close. This eliminates a bad race where an aio_ddp_requeue that happened to run after handle_ddp_close could bump up the active count. Discussed with: jhb@ Sponsored by: Chelsio Communications r349514: cxgbe/t4_tom: the AIO tx job queue must be empty by the time the driver releases the offload resources associated with the tid. Reviewed by: jhb@ Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D20798 r349517: cxgbe/t4_tom: Tweaks to some of the AIO related CTRs. Reviewed by: jhb@ Sponsored by: Chelsio Communications Modified: stable/12/sys/dev/cxgbe/tom/t4_cpl_io.c stable/12/sys/dev/cxgbe/tom/t4_ddp.c stable/12/sys/dev/cxgbe/tom/t4_tom.c Directory Properties: stable/12/ (props changed) Modified: stable/12/sys/dev/cxgbe/tom/t4_cpl_io.c ============================================================================== --- stable/12/sys/dev/cxgbe/tom/t4_cpl_io.c Sat Jul 6 20:31:37 2019 (r349794) +++ stable/12/sys/dev/cxgbe/tom/t4_cpl_io.c Sun Jul 7 00:30:20 2019 (r349795) @@ -722,8 +722,8 @@ t4_push_frames(struct adapter *sc, struct toepcb *toep ("%s: ulp_mode %u for toep %p", __func__, toep->ulp_mode, toep)); #ifdef VERBOSE_TRACES - CTR4(KTR_CXGBE, "%s: tid %d toep flags %#x tp flags %#x drop %d", - __func__, toep->tid, toep->flags, tp->t_flags); + CTR5(KTR_CXGBE, "%s: tid %d toep flags %#x tp flags %#x drop %d", + __func__, toep->tid, toep->flags, tp->t_flags, drop); #endif if (__predict_false(toep->flags & TPF_ABORT_SHUTDOWN)) return; @@ -1242,8 +1242,10 @@ do_peer_close(struct sge_iq *iq, const struct rss_head INP_WLOCK(inp); tp = intotcpcb(inp); - CTR5(KTR_CXGBE, "%s: tid %u (%s), toep_flags 0x%x, inp %p", __func__, - tid, tp ? tcpstates[tp->t_state] : "no tp", toep->flags, inp); + CTR6(KTR_CXGBE, + "%s: tid %u (%s), toep_flags 0x%x, ddp_flags 0x%x, inp %p", + __func__, tid, tp ? tcpstates[tp->t_state] : "no tp", toep->flags, + toep->ddp.flags, inp); if (toep->flags & TPF_ABORT_SHUTDOWN) goto done; @@ -1251,6 +1253,7 @@ do_peer_close(struct sge_iq *iq, const struct rss_head tp->rcv_nxt++; /* FIN */ so = inp->inp_socket; + socantrcvmore(so); if (toep->ulp_mode == ULP_MODE_TCPDDP) { DDP_LOCK(toep); if (__predict_false(toep->ddp.flags & @@ -1258,7 +1261,6 @@ do_peer_close(struct sge_iq *iq, const struct rss_head handle_ddp_close(toep, tp, cpl->rcv_nxt); DDP_UNLOCK(toep); } - socantrcvmore(so); if (toep->ulp_mode != ULP_MODE_RDMA) { KASSERT(tp->rcv_nxt == be32toh(cpl->rcv_nxt), @@ -2223,7 +2225,7 @@ t4_aiotx_queue_toep(struct toepcb *toep) SOCKBUF_LOCK_ASSERT(&toep->inp->inp_socket->so_snd); #ifdef VERBOSE_TRACES CTR3(KTR_CXGBE, "%s: queueing aiotx task for tid %d, active = %s", - __func__, toep->tid, toep->aiotx_task_active ? "true" : "false"); + __func__, toep->tid, toep->aiotx_so != NULL ? "true" : "false"); #endif if (toep->aiotx_task_active) return; @@ -2278,7 +2280,7 @@ t4_aio_queue_aiotx(struct socket *so, struct kaiocb *j SOCKBUF_LOCK(&so->so_snd); #ifdef VERBOSE_TRACES - CTR2(KTR_CXGBE, "%s: queueing %p", __func__, job); + CTR3(KTR_CXGBE, "%s: queueing %p for tid %u", __func__, job, toep->tid); #endif if (!aio_set_cancel_function(job, t4_aiotx_cancel)) panic("new job was cancelled"); Modified: stable/12/sys/dev/cxgbe/tom/t4_ddp.c ============================================================================== --- stable/12/sys/dev/cxgbe/tom/t4_ddp.c Sat Jul 6 20:31:37 2019 (r349794) +++ stable/12/sys/dev/cxgbe/tom/t4_ddp.c Sun Jul 7 00:30:20 2019 (r349795) @@ -220,7 +220,7 @@ release_ddp_resources(struct toepcb *toep) int i; DDP_LOCK(toep); - toep->flags |= DDP_DEAD; + toep->ddp.flags |= DDP_DEAD; for (i = 0; i < nitems(toep->ddp.db); i++) { free_ddp_buffer(toep->td, &toep->ddp.db[i]); } @@ -263,8 +263,8 @@ complete_ddp_buffer(struct toepcb *toep, struct ddp_bu } else toep->ddp.active_id ^= 1; #ifdef VERBOSE_TRACES - CTR2(KTR_CXGBE, "%s: ddp_active_id = %d", __func__, - toep->ddp.active_id); + CTR3(KTR_CXGBE, "%s: tid %u, ddp_active_id = %d", __func__, + toep->tid, toep->ddp.active_id); #endif } else { KASSERT(toep->ddp.active_count != 0 && @@ -534,8 +534,8 @@ handle_ddp_data(struct toepcb *toep, __be32 ddp_report tp->rcv_wnd -= len; #endif #ifdef VERBOSE_TRACES - CTR4(KTR_CXGBE, "%s: DDP[%d] placed %d bytes (%#x)", __func__, db_idx, - len, report); + CTR5(KTR_CXGBE, "%s: tid %u, DDP[%d] placed %d bytes (%#x)", __func__, + toep->tid, db_idx, len, report); #endif /* receive buffer autosize */ @@ -573,8 +573,9 @@ handle_ddp_data(struct toepcb *toep, __be32 ddp_report } else { copied = job->aio_received; #ifdef VERBOSE_TRACES - CTR4(KTR_CXGBE, "%s: completing %p (copied %ld, placed %d)", - __func__, job, copied, len); + CTR5(KTR_CXGBE, + "%s: tid %u, completing %p (copied %ld, placed %d)", + __func__, toep->tid, job, copied, len); #endif aio_complete(job, copied + len, 0); t4_rcvd(&toep->td->tod, tp); @@ -1791,8 +1792,9 @@ sbcopy: } #ifdef VERBOSE_TRACES - CTR5(KTR_CXGBE, "%s: scheduling %p for DDP[%d] (flags %#lx/%#lx)", - __func__, job, db_idx, ddp_flags, ddp_flags_mask); + CTR6(KTR_CXGBE, + "%s: tid %u, scheduling %p for DDP[%d] (flags %#lx/%#lx)", __func__, + toep->tid, job, db_idx, ddp_flags, ddp_flags_mask); #endif /* Give the chip the go-ahead. */ t4_wrq_tx(sc, wr); @@ -1918,7 +1920,7 @@ t4_aio_queue_ddp(struct socket *so, struct kaiocb *job */ #ifdef VERBOSE_TRACES - CTR2(KTR_CXGBE, "%s: queueing %p", __func__, job); + CTR3(KTR_CXGBE, "%s: queueing %p for tid %u", __func__, job, toep->tid); #endif if (!aio_set_cancel_function(job, t4_aio_cancel_queued)) panic("new job was cancelled"); Modified: stable/12/sys/dev/cxgbe/tom/t4_tom.c ============================================================================== --- stable/12/sys/dev/cxgbe/tom/t4_tom.c Sat Jul 6 20:31:37 2019 (r349794) +++ stable/12/sys/dev/cxgbe/tom/t4_tom.c Sun Jul 7 00:30:20 2019 (r349795) @@ -294,6 +294,7 @@ release_offload_resources(struct toepcb *toep) if (toep->ulp_mode == ULP_MODE_TCPDDP) ddp_assert_empty(toep); #endif + MPASS(TAILQ_EMPTY(&toep->aiotx_jobq)); if (toep->l2te) t4_l2t_release(toep->l2te);