From owner-svn-src-head@freebsd.org Sun Feb 10 20:14:00 2019 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4FF6814DA040; Sun, 10 Feb 2019 20:14:00 +0000 (UTC) (envelope-from nwhitehorn@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E70F08F2DB; Sun, 10 Feb 2019 20:13:59 +0000 (UTC) (envelope-from nwhitehorn@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id D5EAE1D721; Sun, 10 Feb 2019 20:13:59 +0000 (UTC) (envelope-from nwhitehorn@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x1AKDx7f019846; Sun, 10 Feb 2019 20:13:59 GMT (envelope-from nwhitehorn@FreeBSD.org) Received: (from nwhitehorn@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x1AKDxt1019844; Sun, 10 Feb 2019 20:13:59 GMT (envelope-from nwhitehorn@FreeBSD.org) Message-Id: <201902102013.x1AKDxt1019844@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: nwhitehorn set sender to nwhitehorn@FreeBSD.org using -f From: Nathan Whitehorn Date: Sun, 10 Feb 2019 20:13:59 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r343969 - head/sys/mips/cavium/octe X-SVN-Group: head X-SVN-Commit-Author: nwhitehorn X-SVN-Commit-Paths: head/sys/mips/cavium/octe X-SVN-Commit-Revision: 343969 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: E70F08F2DB X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.996,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.982,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Feb 2019 20:14:00 -0000 Author: nwhitehorn Date: Sun Feb 10 20:13:59 2019 New Revision: 343969 URL: https://svnweb.freebsd.org/changeset/base/343969 Log: Performance improvements for octe(4): - Distribute RX load across multiple cores, if present. This reverts r217212, which is no longer relevant (I think because of the newer SDK). - Use newer APIs for pinning taskqueue entries to specific cores. - Deepen RX buffers. This more than doubles NAT forwarding throughput on my EdgeRouter Lite from, with typical packet mixture, 90 Mbps to over 200 Mbps. The result matches forwarding throughput in Linux without the UBNT hardware offload on the same hardware, and thus likely reflects hardware limits. Reviewed by: jhibbits Modified: head/sys/mips/cavium/octe/ethernet-defines.h head/sys/mips/cavium/octe/ethernet-rx.c head/sys/mips/cavium/octe/ethernet.c Modified: head/sys/mips/cavium/octe/ethernet-defines.h ============================================================================== --- head/sys/mips/cavium/octe/ethernet-defines.h Sun Feb 10 19:20:03 2019 (r343968) +++ head/sys/mips/cavium/octe/ethernet-defines.h Sun Feb 10 20:13:59 2019 (r343969) @@ -38,14 +38,14 @@ AND WITH ALL FAULTS AND CAVIUM NETWORKS MAKES NO PROM * the driver uses the default from below. */ -#define INTERRUPT_LIMIT 10000 /* Max interrupts per second per core */ +#define INTERRUPT_LIMIT 1000 /* Max interrupts per second per core */ /*#define INTERRUPT_LIMIT 0 *//* Don't limit the number of interrupts */ #define USE_RED 1 /* Enable Random Early Dropping under load */ #define USE_10MBPS_PREAMBLE_WORKAROUND 1 /* Allow SW based preamble removal at 10Mbps to workaround PHYs giving us bad preambles */ #define DONT_WRITEBACK(x) (x) /* Use this to have all FPA frees also tell the L2 not to write data to memory */ /*#define DONT_WRITEBACK(x) 0 *//* Use this to not have FPA frees control L2 */ -#define MAX_RX_PACKETS 120 /* Maximum number of packets to process per interrupt. */ +#define MAX_RX_PACKETS 1024 /* Maximum number of packets to process per interrupt. */ #define MAX_OUT_QUEUE_DEPTH 1000 #define FAU_NUM_PACKET_BUFFERS_TO_FREE (CVMX_FAU_REG_END - sizeof(uint32_t)) Modified: head/sys/mips/cavium/octe/ethernet-rx.c ============================================================================== --- head/sys/mips/cavium/octe/ethernet-rx.c Sun Feb 10 19:20:03 2019 (r343968) +++ head/sys/mips/cavium/octe/ethernet-rx.c Sun Feb 10 20:13:59 2019 (r343969) @@ -57,8 +57,6 @@ extern struct ifnet *cvm_oct_device[]; static struct task cvm_oct_task; static struct taskqueue *cvm_oct_taskq; -static int cvm_oct_rx_active; - /** * Interrupt handler. The interrupt occurs whenever the POW * transitions from 0->1 packets in our group. @@ -77,10 +75,9 @@ int cvm_oct_do_interrupt(void *dev_id) cvmx_write_csr(CVMX_POW_WQ_INT, 0x10001<