From owner-svn-src-all@freebsd.org Tue May 26 16:05:42 2020 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9C7262CC3FA; Tue, 26 May 2020 16:05:42 +0000 (UTC) (envelope-from mw@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49Wf2G3hGxz4RYH; Tue, 26 May 2020 16:05:42 +0000 (UTC) (envelope-from mw@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 79B19260BD; Tue, 26 May 2020 16:05:42 +0000 (UTC) (envelope-from mw@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 04QG5gv6083144; Tue, 26 May 2020 16:05:42 GMT (envelope-from mw@FreeBSD.org) Received: (from mw@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 04QG5g10083143; Tue, 26 May 2020 16:05:42 GMT (envelope-from mw@FreeBSD.org) Message-Id: <202005261605.04QG5g10083143@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mw set sender to mw@FreeBSD.org using -f From: Marcin Wojtas Date: Tue, 26 May 2020 16:05:42 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r361529 - head/sys/dev/ena X-SVN-Group: head X-SVN-Commit-Author: mw X-SVN-Commit-Paths: head/sys/dev/ena X-SVN-Commit-Revision: 361529 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 May 2020 16:05:42 -0000 Author: mw Date: Tue May 26 16:05:42 2020 New Revision: 361529 URL: https://svnweb.freebsd.org/changeset/base/361529 Log: Refactor ena_tx_map_mbuf() function There is no guarantee from bus_dmamap_load_mbuf_sg() for matching mbuf chain segments to dma physical segments. This patch ensure correctly mapping to LLQ header and DMA segments. Submitted by: Ido Segev Obtained from: Amazon, Inc. Modified: head/sys/dev/ena/ena_datapath.c Modified: head/sys/dev/ena/ena_datapath.c ============================================================================== --- head/sys/dev/ena/ena_datapath.c Tue May 26 16:02:10 2020 (r361528) +++ head/sys/dev/ena/ena_datapath.c Tue May 26 16:05:42 2020 (r361529) @@ -812,9 +812,8 @@ ena_tx_map_mbuf(struct ena_ring *tx_ring, struct ena_t struct ena_com_buf *ena_buf; bus_dma_segment_t segs[ENA_BUS_DMA_SEGS]; size_t iseg = 0; - uint32_t mbuf_head_len, frag_len; - uint16_t push_len = 0; - uint16_t delta = 0; + uint32_t mbuf_head_len; + uint16_t offset; int rc, nsegs; mbuf_head_len = mbuf->m_len; @@ -833,7 +832,6 @@ ena_tx_map_mbuf(struct ena_ring *tx_ring, struct ena_t goto dma_error; } - if (tx_ring->tx_mem_queue_type == ENA_ADMIN_PLACEMENT_POLICY_DEV) { /* * When the device is LLQ mode, the driver will copy @@ -845,44 +843,48 @@ ena_tx_map_mbuf(struct ena_ring *tx_ring, struct ena_t * First check if header fits in the mbuf. If not, copy it to * separate buffer that will be holding linearized data. */ - push_len = min_t(uint32_t, mbuf->m_pkthdr.len, - tx_ring->tx_max_header_size); - *header_len = push_len; + *header_len = min_t(uint32_t, mbuf->m_pkthdr.len, tx_ring->tx_max_header_size); + /* If header is in linear space, just point into mbuf's data. */ - if (likely(push_len <= mbuf_head_len)) { + if (likely(*header_len <= mbuf_head_len)) { *push_hdr = mbuf->m_data; /* * Otherwise, copy whole portion of header from multiple mbufs * to intermediate buffer. */ } else { - m_copydata(mbuf, 0, push_len, - tx_ring->push_buf_intermediate_buf); + m_copydata(mbuf, 0, *header_len, tx_ring->push_buf_intermediate_buf); *push_hdr = tx_ring->push_buf_intermediate_buf; counter_u64_add(tx_ring->tx_stats.llq_buffer_copy, 1); - delta = push_len - mbuf_head_len; } ena_trace(ENA_DBG | ENA_TXPTH, "mbuf: %p header_buf->vaddr: %p push_len: %d\n", - mbuf, *push_hdr, push_len); + mbuf, *push_hdr, *header_len); - /* - * If header was in linear memory space, map for the dma rest of the data - * in the first mbuf of the mbuf chain. - */ - if (mbuf_head_len > push_len) { - ena_buf->paddr = segs[iseg].ds_addr + push_len; - ena_buf->len = segs[iseg].ds_len - push_len; - ena_buf++; - tx_info->num_of_bufs++; + /* If packet is fitted in LLQ header, no need for DMA segments. */ + if (mbuf->m_pkthdr.len <= tx_ring->tx_max_header_size) { + return (0); + } else { + offset = tx_ring->tx_max_header_size; + /* + * As Header part is mapped to LLQ header, we can skip it and just + * map the residuum of the mbuf to DMA Segments. + */ + while (offset > 0) { + if (offset >= segs[iseg].ds_len) { + offset -= segs[iseg].ds_len; + } else { + ena_buf->paddr = segs[iseg].ds_addr + offset; + ena_buf->len = segs[iseg].ds_len - offset; + ena_buf++; + tx_info->num_of_bufs++; + offset = 0; + } + iseg++; + } } - /* - * Advance the seg index as either the 1st mbuf was mapped or is - * a part of push_hdr. - */ - iseg++; } else { *push_hdr = NULL; /* @@ -893,41 +895,6 @@ ena_tx_map_mbuf(struct ena_ring *tx_ring, struct ena_t * header on it's own. */ *header_len = 0; - } - - /* - * If header is in non linear space (delta > 0), then skip mbufs - * containing header and map the last one containing both header and the - * packet data. - * The first segment is already counted in. - * If LLQ is not supported, the loop will be skipped. - */ - while (delta > 0) { - frag_len = segs[iseg].ds_len; - - /* - * If whole segment contains header just move to the - * next one and reduce delta. - */ - if (unlikely(delta >= frag_len)) { - delta -= frag_len; - } else { - /* - * Map rest of the packet data that was contained in - * the mbuf. - */ - ena_buf->paddr = segs[iseg].ds_addr + delta; - ena_buf->len = frag_len - delta; - ena_buf++; - tx_info->num_of_bufs++; - - delta = 0; - } - iseg++; - } - - if (mbuf == NULL) { - return (0); } /* Map rest of the mbuf */