From owner-svn-src-releng@freebsd.org Tue Jun 9 16:10:58 2020 Return-Path: Delivered-To: svn-src-releng@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 15D13331B18; Tue, 9 Jun 2020 16:10:58 +0000 (UTC) (envelope-from gordon@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49hFTs6p93z3gjr; Tue, 9 Jun 2020 16:10:57 +0000 (UTC) (envelope-from gordon@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E4ACA17FE8; Tue, 9 Jun 2020 16:10:57 +0000 (UTC) (envelope-from gordon@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 059GAv9M084942; Tue, 9 Jun 2020 16:10:57 GMT (envelope-from gordon@FreeBSD.org) Received: (from gordon@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 059GAv1w084940; Tue, 9 Jun 2020 16:10:57 GMT (envelope-from gordon@FreeBSD.org) Message-Id: <202006091610.059GAv1w084940@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: gordon set sender to gordon@FreeBSD.org using -f From: Gordon Tetlow Date: Tue, 9 Jun 2020 16:10:57 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-releng@freebsd.org Subject: svn commit: r361970 - releng/12.1/sys/dev/ena X-SVN-Group: releng X-SVN-Commit-Author: gordon X-SVN-Commit-Paths: releng/12.1/sys/dev/ena X-SVN-Commit-Revision: 361970 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-releng@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: SVN commit messages for the release engineering / security commits to the src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Jun 2020 16:10:58 -0000 Author: gordon Date: Tue Jun 9 16:10:57 2020 New Revision: 361970 URL: https://svnweb.freebsd.org/changeset/base/361970 Log: Fix stability issues in ena(4) driver. Approved by: so Security: FreeBSD-EN-20:11.ena Modified: releng/12.1/sys/dev/ena/ena.c releng/12.1/sys/dev/ena/ena.h Modified: releng/12.1/sys/dev/ena/ena.c ============================================================================== --- releng/12.1/sys/dev/ena/ena.c Tue Jun 9 16:06:10 2020 (r361969) +++ releng/12.1/sys/dev/ena/ena.c Tue Jun 9 16:10:57 2020 (r361970) @@ -200,6 +200,19 @@ int ena_log_level = ENA_ALERT | ENA_WARNING; SYSCTL_INT(_hw_ena, OID_AUTO, log_level, CTLFLAG_RWTUN, &ena_log_level, 0, "Logging level indicating verbosity of the logs"); +/* + * Use 9k mbufs for the Rx buffers. Default to 0 (use page size mbufs instead). + * Using 9k mbufs in low memory conditions might cause allocation to take a lot + * of time and lead to the OS instability as it needs to look for the contiguous + * pages. + * However, page size mbufs has a bit smaller throughput than 9k mbufs, so if + * the network performance is the priority, the 9k mbufs can be used. + */ +int ena_enable_9k_mbufs = 0; +SYSCTL_INT(_hw_ena, OID_AUTO, enable_9k_mbufs, CTLFLAG_RDTUN, + &ena_enable_9k_mbufs, 0, "Use 9 kB mbufs for Rx descriptors"); +#define ena_mbuf_sz (ena_enable_9k_mbufs ? MJUM9BYTES : MJUMPAGESIZE) + static ena_vendor_info_t ena_vendor_info_array[] = { { PCI_VENDOR_ID_AMAZON, PCI_DEV_ID_ENA_PF, 0}, { PCI_VENDOR_ID_AMAZON, PCI_DEV_ID_ENA_LLQ_PF, 0}, @@ -470,6 +483,7 @@ ena_init_io_rings(struct ena_adapter *adapter) rxr->que = que; rxr->empty_rx_queue = 0; + rxr->rx_mbuf_sz = ena_mbuf_sz; } } @@ -548,9 +562,9 @@ ena_setup_rx_dma_tag(struct ena_adapter *adapter) ENA_DMA_BIT_MASK(adapter->dma_width), /* lowaddr of excl window */ BUS_SPACE_MAXADDR, /* highaddr of excl window */ NULL, NULL, /* filter, filterarg */ - MJUM16BYTES, /* maxsize */ + ena_mbuf_sz, /* maxsize */ adapter->max_rx_sgl_size, /* nsegments */ - MJUM16BYTES, /* maxsegsize */ + ena_mbuf_sz, /* maxsegsize */ 0, /* flags */ NULL, /* lockfunc */ NULL, /* lockarg */ @@ -957,7 +971,8 @@ ena_alloc_rx_mbuf(struct ena_adapter *adapter, return (0); /* Get mbuf using UMA allocator */ - rx_info->mbuf = m_getjcl(M_NOWAIT, MT_DATA, M_PKTHDR, MJUM16BYTES); + rx_info->mbuf = m_getjcl(M_NOWAIT, MT_DATA, M_PKTHDR, + rx_ring->rx_mbuf_sz); if (unlikely(rx_info->mbuf == NULL)) { counter_u64_add(rx_ring->rx_stats.mjum_alloc_fail, 1); @@ -968,7 +983,7 @@ ena_alloc_rx_mbuf(struct ena_adapter *adapter, } mlen = MCLBYTES; } else { - mlen = MJUM16BYTES; + mlen = rx_ring->rx_mbuf_sz; } /* Set mbuf length*/ rx_info->mbuf->m_pkthdr.len = rx_info->mbuf->m_len = mlen; Modified: releng/12.1/sys/dev/ena/ena.h ============================================================================== --- releng/12.1/sys/dev/ena/ena.h Tue Jun 9 16:06:10 2020 (r361969) +++ releng/12.1/sys/dev/ena/ena.h Tue Jun 9 16:10:57 2020 (r361970) @@ -41,7 +41,7 @@ #define DRV_MODULE_VER_MAJOR 0 #define DRV_MODULE_VER_MINOR 8 -#define DRV_MODULE_VER_SUBMINOR 4 +#define DRV_MODULE_VER_SUBMINOR 5 #define DRV_MODULE_NAME "ena" @@ -238,8 +238,12 @@ struct ena_ring { /* Determines if device will use LLQ or normal mode for TX */ enum ena_admin_placement_policy_type tx_mem_queue_type; - /* The maximum length the driver can push to the device (For LLQ) */ - uint8_t tx_max_header_size; + union { + /* The maximum length the driver can push to the device (For LLQ) */ + uint8_t tx_max_header_size; + /* The maximum (and default) mbuf size for the Rx descriptor. */ + uint16_t rx_mbuf_sz; + }; struct ena_com_rx_buf_info ena_bufs[ENA_PKT_MAX_BUFS];