From owner-svn-src-all@freebsd.org Wed May 11 13:38:30 2016 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CADE4B363CB; Wed, 11 May 2016 13:38:30 +0000 (UTC) (envelope-from zbb@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 887521675; Wed, 11 May 2016 13:38:30 +0000 (UTC) (envelope-from zbb@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id u4BDcTv0073873; Wed, 11 May 2016 13:38:29 GMT (envelope-from zbb@FreeBSD.org) Received: (from zbb@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id u4BDcTwO073872; Wed, 11 May 2016 13:38:29 GMT (envelope-from zbb@FreeBSD.org) Message-Id: <201605111338.u4BDcTwO073872@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: zbb set sender to zbb@FreeBSD.org using -f From: Zbigniew Bodek Date: Wed, 11 May 2016 13:38:29 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r299446 - head/sys/dev/vnic X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 May 2016 13:38:30 -0000 Author: zbb Date: Wed May 11 13:38:29 2016 New Revision: 299446 URL: https://svnweb.freebsd.org/changeset/base/299446 Log: Fix deadlock in VNIC when using single CPU only Number of free Tx descriptors does not need to be locked since it can be modified atomically between SND and CQ tasks. It will also block Tx routine from sending packets while CQ will not be able to free descriptors. Obtained from: Semihalf Sponsored by: Cavium Differential Revision: https://reviews.freebsd.org/D6266 Modified: head/sys/dev/vnic/nicvf_queues.c Modified: head/sys/dev/vnic/nicvf_queues.c ============================================================================== --- head/sys/dev/vnic/nicvf_queues.c Wed May 11 13:23:56 2016 (r299445) +++ head/sys/dev/vnic/nicvf_queues.c Wed May 11 13:38:29 2016 (r299446) @@ -48,6 +48,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include #include @@ -691,7 +692,7 @@ nicvf_rcv_pkt_handler(struct nicvf *nic, return (0); } -static int +static void nicvf_snd_pkt_handler(struct nicvf *nic, struct cmp_queue *cq, struct cqe_send_t *cqe_tx, int cqe_type) { @@ -702,15 +703,10 @@ nicvf_snd_pkt_handler(struct nicvf *nic, mbuf = NULL; sq = &nic->qs->sq[cqe_tx->sq_idx]; - /* Avoid blocking here since we hold a non-sleepable NICVF_CMP_LOCK */ - if (NICVF_TX_TRYLOCK(sq) == 0) - return (EAGAIN); hdr = (struct sq_hdr_subdesc *)GET_SQ_DESC(sq, cqe_tx->sqe_ptr); - if (hdr->subdesc_type != SQ_DESC_TYPE_HEADER) { - NICVF_TX_UNLOCK(sq); - return (0); - } + if (hdr->subdesc_type != SQ_DESC_TYPE_HEADER) + return; dprintf(nic->dev, "%s Qset #%d SQ #%d SQ ptr #%d subdesc count %d\n", @@ -728,9 +724,6 @@ nicvf_snd_pkt_handler(struct nicvf *nic, } nicvf_check_cqe_tx_errs(nic, cq, cqe_tx); - - NICVF_TX_UNLOCK(sq); - return (0); } static int @@ -788,16 +781,8 @@ nicvf_cq_intr_handler(struct nicvf *nic, work_done++; break; case CQE_TYPE_SEND: - cmp_err = nicvf_snd_pkt_handler(nic, cq, - (void *)cq_desc, CQE_TYPE_SEND); - if (__predict_false(cmp_err != 0)) { - /* - * Ups. Cannot finish now. - * Let's try again later. - */ - goto done; - } - + nicvf_snd_pkt_handler(nic, cq, (void *)cq_desc, + CQE_TYPE_SEND); tx_done++; break; case CQE_TYPE_INVALID: @@ -1085,7 +1070,7 @@ nicvf_init_snd_queue(struct nicvf *nic, sq->desc = sq->dmem.base; sq->head = sq->tail = 0; - sq->free_cnt = q_len - 1; + atomic_store_rel_int(&sq->free_cnt, q_len - 1); sq->thresh = SND_QUEUE_THRESH; sq->idx = qidx; sq->nic = nic; @@ -1676,7 +1661,7 @@ nicvf_get_sq_desc(struct snd_queue *sq, int qentry; qentry = sq->tail; - sq->free_cnt -= desc_cnt; + atomic_subtract_int(&sq->free_cnt, desc_cnt); sq->tail += desc_cnt; sq->tail &= (sq->dmem.q_len - 1); @@ -1688,7 +1673,7 @@ static void nicvf_put_sq_desc(struct snd_queue *sq, int desc_cnt) { - sq->free_cnt += desc_cnt; + atomic_add_int(&sq->free_cnt, desc_cnt); sq->head += desc_cnt; sq->head &= (sq->dmem.q_len - 1); }