From owner-svn-src-head@freebsd.org Sat Sep 9 05:12:15 2017 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DCE98E03486; Sat, 9 Sep 2017 05:12:15 +0000 (UTC) (envelope-from np@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A6A407FB4F; Sat, 9 Sep 2017 05:12:15 +0000 (UTC) (envelope-from np@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v895CEZ5031634; Sat, 9 Sep 2017 05:12:14 GMT (envelope-from np@FreeBSD.org) Received: (from np@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v895CEpW031633; Sat, 9 Sep 2017 05:12:14 GMT (envelope-from np@FreeBSD.org) Message-Id: <201709090512.v895CEpW031633@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: np set sender to np@FreeBSD.org using -f From: Navdeep Parhar Date: Sat, 9 Sep 2017 05:12:14 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r323343 - head/sys/dev/cxgbe X-SVN-Group: head X-SVN-Commit-Author: np X-SVN-Commit-Paths: head/sys/dev/cxgbe X-SVN-Commit-Revision: 323343 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Sep 2017 05:12:16 -0000 Author: np Date: Sat Sep 9 05:12:14 2017 New Revision: 323343 URL: https://svnweb.freebsd.org/changeset/base/323343 Log: cxgbe(4): Fix a couple of problems in the sge_wrq data path. - start_wrq_wr must not drain the wr_list if there are incomplete_wrs pending. This can happen when a t4_wrq_tx runs between two start_wrq_wr. - commit_wrq_wr must examine the cookie's pidx and ndesc with the queue's lock held. Otherwise there is a bad race when incomplete WRs are being completed and commit_wrq_wr for the WR that is ahead in the queue updates the next incomplete WR's cookie's pidx/ndesc but the commit_wrq_wr for the second one is using stale values that it read without the lock. MFC after: 1 week Sponsored by: Chelsio Communications Modified: head/sys/dev/cxgbe/t4_sge.c Modified: head/sys/dev/cxgbe/t4_sge.c ============================================================================== --- head/sys/dev/cxgbe/t4_sge.c Sat Sep 9 03:09:02 2017 (r323342) +++ head/sys/dev/cxgbe/t4_sge.c Sat Sep 9 05:12:14 2017 (r323343) @@ -2354,7 +2354,7 @@ start_wrq_wr(struct sge_wrq *wrq, int len16, struct wr EQ_LOCK(eq); - if (!STAILQ_EMPTY(&wrq->wr_list)) + if (TAILQ_EMPTY(&wrq->incomplete_wrs) && !STAILQ_EMPTY(&wrq->wr_list)) drain_wrq_wr_list(sc, wrq); if (!STAILQ_EMPTY(&wrq->wr_list)) { @@ -2408,9 +2408,6 @@ commit_wrq_wr(struct sge_wrq *wrq, void *w, struct wrq return; } - ndesc = cookie->ndesc; /* Can be more than SGE_MAX_WR_NDESC here. */ - pidx = cookie->pidx; - MPASS(pidx >= 0 && pidx < eq->sidx); if (__predict_false(w == &wrq->ss[0])) { int n = (eq->sidx - wrq->ss_pidx) * EQ_ESIZE; @@ -2422,6 +2419,9 @@ commit_wrq_wr(struct sge_wrq *wrq, void *w, struct wrq wrq->tx_wrs_direct++; EQ_LOCK(eq); + ndesc = cookie->ndesc; /* Can be more than SGE_MAX_WR_NDESC here. */ + pidx = cookie->pidx; + MPASS(pidx >= 0 && pidx < eq->sidx); prev = TAILQ_PREV(cookie, wrq_incomplete_wrs, link); next = TAILQ_NEXT(cookie, link); if (prev == NULL) {