From owner-svn-src-all@FreeBSD.ORG Fri Feb 27 07:39:12 2015 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 14792738; Fri, 27 Feb 2015 07:39:12 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F33C9FED; Fri, 27 Feb 2015 07:39:11 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.9/8.14.9) with ESMTP id t1R7dBhf016172; Fri, 27 Feb 2015 07:39:11 GMT (envelope-from arybchik@FreeBSD.org) Received: (from arybchik@localhost) by svn.freebsd.org (8.14.9/8.14.9/Submit) id t1R7dAiK016165; Fri, 27 Feb 2015 07:39:10 GMT (envelope-from arybchik@FreeBSD.org) Message-Id: <201502270739.t1R7dAiK016165@svn.freebsd.org> X-Authentication-Warning: svn.freebsd.org: arybchik set sender to arybchik@FreeBSD.org using -f From: Andrew Rybchenko Date: Fri, 27 Feb 2015 07:39:10 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r279351 - head/sys/dev/sfxge X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Feb 2015 07:39:12 -0000 Author: arybchik Date: Fri Feb 27 07:39:09 2015 New Revision: 279351 URL: https://svnweb.freebsd.org/changeset/base/279351 Log: sfxge: expect required init_state on data path and in periodic calls With the patch applied the number of instruction events is 1% less and number of mispredicted branch events is 5% less under multistream TCP traffic load close to line rate. Sponsored by: Solarflare Communications, Inc. Approved by: gnn (mentor) Modified: head/sys/dev/sfxge/sfxge_ev.c head/sys/dev/sfxge/sfxge_intr.c head/sys/dev/sfxge/sfxge_port.c head/sys/dev/sfxge/sfxge_rx.c head/sys/dev/sfxge/sfxge_tx.c Modified: head/sys/dev/sfxge/sfxge_ev.c ============================================================================== --- head/sys/dev/sfxge/sfxge_ev.c Fri Feb 27 04:45:47 2015 (r279350) +++ head/sys/dev/sfxge/sfxge_ev.c Fri Feb 27 07:39:09 2015 (r279351) @@ -103,7 +103,7 @@ sfxge_ev_rx(void *arg, uint32_t label, u KASSERT(evq->index == rxq->index, ("evq->index != rxq->index")); - if (rxq->init_state != SFXGE_RXQ_STARTED) + if (__predict_false(rxq->init_state != SFXGE_RXQ_STARTED)) goto done; expected = rxq->pending++ & rxq->ptr_mask; @@ -256,7 +256,7 @@ sfxge_ev_tx(void *arg, uint32_t label, u KASSERT(evq->index == txq->evq_index, ("evq->index != txq->evq_index")); - if (txq->init_state != SFXGE_TXQ_STARTED) + if (__predict_false(txq->init_state != SFXGE_TXQ_STARTED)) goto done; stop = (id + 1) & txq->ptr_mask; @@ -433,7 +433,7 @@ sfxge_ev_stat_update(struct sfxge_softc SFXGE_ADAPTER_LOCK(sc); - if (sc->evq[0]->init_state != SFXGE_EVQ_STARTED) + if (__predict_false(sc->evq[0]->init_state != SFXGE_EVQ_STARTED)) goto out; now = ticks; @@ -598,8 +598,8 @@ sfxge_ev_qpoll(struct sfxge_evq *evq) SFXGE_EVQ_LOCK(evq); - if (evq->init_state != SFXGE_EVQ_STARTING && - evq->init_state != SFXGE_EVQ_STARTED) { + if (__predict_false(evq->init_state != SFXGE_EVQ_STARTING && + evq->init_state != SFXGE_EVQ_STARTED)) { rc = EINVAL; goto fail; } Modified: head/sys/dev/sfxge/sfxge_intr.c ============================================================================== --- head/sys/dev/sfxge/sfxge_intr.c Fri Feb 27 04:45:47 2015 (r279350) +++ head/sys/dev/sfxge/sfxge_intr.c Fri Feb 27 07:39:09 2015 (r279351) @@ -134,7 +134,7 @@ sfxge_intr_message(void *arg) KASSERT(intr->type == EFX_INTR_MESSAGE, ("intr->type != EFX_INTR_MESSAGE")); - if (intr->state != SFXGE_INTR_STARTED) + if (__predict_false(intr->state != SFXGE_INTR_STARTED)) return; (void)efx_intr_status_message(enp, index, &fatal); Modified: head/sys/dev/sfxge/sfxge_port.c ============================================================================== --- head/sys/dev/sfxge/sfxge_port.c Fri Feb 27 04:45:47 2015 (r279350) +++ head/sys/dev/sfxge/sfxge_port.c Fri Feb 27 07:39:09 2015 (r279351) @@ -50,7 +50,7 @@ sfxge_mac_stat_update(struct sfxge_softc SFXGE_PORT_LOCK_ASSERT_OWNED(port); - if (port->init_state != SFXGE_PORT_STARTED) { + if (__predict_false(port->init_state != SFXGE_PORT_STARTED)) { rc = 0; goto out; } @@ -241,7 +241,7 @@ sfxge_port_wanted_fc_handler(SYSCTL_HAND SFXGE_PORT_LOCK(port); if (port->wanted_fc != fcntl) { - if (port->init_state == SFXGE_PORT_STARTED) + if (__predict_false(port->init_state == SFXGE_PORT_STARTED)) error = efx_mac_fcntl_set(sc->enp, port->wanted_fc, B_TRUE); @@ -272,7 +272,8 @@ sfxge_port_link_fc_handler(SYSCTL_HANDLE port = &sc->port; SFXGE_PORT_LOCK(port); - if (port->init_state == SFXGE_PORT_STARTED && SFXGE_LINK_UP(sc)) + if (__predict_true(port->init_state == SFXGE_PORT_STARTED) && + SFXGE_LINK_UP(sc)) efx_mac_fcntl_get(sc->enp, &wanted_fc, &link_fc); else link_fc = 0; @@ -327,7 +328,7 @@ sfxge_mac_poll_work(void *arg, int npend SFXGE_PORT_LOCK(port); - if (port->init_state != SFXGE_PORT_STARTED) + if (__predict_false(port->init_state != SFXGE_PORT_STARTED)) goto done; /* This may sleep waiting for MCDI completion */ @@ -394,7 +395,7 @@ sfxge_mac_filter_set(struct sfxge_softc * lock is held in sleeping thread. Both problems are repeatable * on LAG with LACP proto bring up. */ - if (port->init_state == SFXGE_PORT_STARTED) + if (__predict_true(port->init_state == SFXGE_PORT_STARTED)) rc = sfxge_mac_filter_set_locked(sc); else rc = 0; @@ -518,7 +519,7 @@ sfxge_phy_stat_update(struct sfxge_softc SFXGE_PORT_LOCK_ASSERT_OWNED(port); - if (port->init_state != SFXGE_PORT_STARTED) { + if (__predict_false(port->init_state != SFXGE_PORT_STARTED)) { rc = 0; goto out; } Modified: head/sys/dev/sfxge/sfxge_rx.c ============================================================================== --- head/sys/dev/sfxge/sfxge_rx.c Fri Feb 27 04:45:47 2015 (r279350) +++ head/sys/dev/sfxge/sfxge_rx.c Fri Feb 27 07:39:09 2015 (r279351) @@ -209,7 +209,7 @@ sfxge_rx_qfill(struct sfxge_rxq *rxq, un SFXGE_EVQ_LOCK_ASSERT_OWNED(evq); - if (rxq->init_state != SFXGE_RXQ_STARTED) + if (__predict_false(rxq->init_state != SFXGE_RXQ_STARTED)) return; rxfill = rxq->added - rxq->completed; @@ -269,7 +269,7 @@ void sfxge_rx_qrefill(struct sfxge_rxq *rxq) { - if (rxq->init_state != SFXGE_RXQ_STARTED) + if (__predict_false(rxq->init_state != SFXGE_RXQ_STARTED)) return; /* Make sure the queue is full */ @@ -760,7 +760,7 @@ sfxge_rx_qcomplete(struct sfxge_rxq *rxq rx_desc = &rxq->queue[id]; m = rx_desc->mbuf; - if (rxq->init_state != SFXGE_RXQ_STARTED) + if (__predict_false(rxq->init_state != SFXGE_RXQ_STARTED)) goto discard; if (rx_desc->flags & (EFX_ADDR_MISMATCH | EFX_DISCARD)) Modified: head/sys/dev/sfxge/sfxge_tx.c ============================================================================== --- head/sys/dev/sfxge/sfxge_tx.c Fri Feb 27 04:45:47 2015 (r279350) +++ head/sys/dev/sfxge/sfxge_tx.c Fri Feb 27 07:39:09 2015 (r279351) @@ -311,7 +311,7 @@ static int sfxge_tx_queue_mbuf(struct sf if (mbuf->m_pkthdr.csum_flags & CSUM_TSO) prefetch_read_many(mbuf->m_data); - if (txq->init_state != SFXGE_TXQ_STARTED) { + if (__predict_false(txq->init_state != SFXGE_TXQ_STARTED)) { rc = EINTR; goto reject; } @@ -1143,7 +1143,7 @@ sfxge_tx_qunblock(struct sfxge_txq *txq) SFXGE_EVQ_LOCK_ASSERT_OWNED(evq); - if (txq->init_state != SFXGE_TXQ_STARTED) + if (__predict_false(txq->init_state != SFXGE_TXQ_STARTED)) return; SFXGE_TXQ_LOCK(txq);