From owner-svn-src-all@FreeBSD.ORG Thu Jan 19 09:36:19 2012 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E0753106564A; Thu, 19 Jan 2012 09:36:19 +0000 (UTC) (envelope-from luigi@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id CAFEC8FC08; Thu, 19 Jan 2012 09:36:19 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.4/8.14.4) with ESMTP id q0J9aJH4080857; Thu, 19 Jan 2012 09:36:19 GMT (envelope-from luigi@svn.freebsd.org) Received: (from luigi@localhost) by svn.freebsd.org (8.14.4/8.14.4/Submit) id q0J9aJX8080855; Thu, 19 Jan 2012 09:36:19 GMT (envelope-from luigi@svn.freebsd.org) Message-Id: <201201190936.q0J9aJX8080855@svn.freebsd.org> From: Luigi Rizzo Date: Thu, 19 Jan 2012 09:36:19 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r230329 - head/sys/dev/ixgbe X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jan 2012 09:36:20 -0000 Author: luigi Date: Thu Jan 19 09:36:19 2012 New Revision: 230329 URL: http://svn.freebsd.org/changeset/base/230329 Log: netmap-related changes: 1. correct the initialization of RDT when there is an ixgbe_init() while a netmap client is active. This code was previously in ixgbe_initialize_receive_units() but RDT is overwritten shortly afterwards in ixgbe_init_locked() 2. add code (not active yet) to disable CRCSTRIP while in netmap mode. From all evidence i could gather, it seems that when the 82599 has to write a data block that is not a full cache line, it first reads the line (64 bytes) and then writes back the updated version. This hurts reception of min-sized frames, which are only 60 bytes if the CRC is stripped: i could never get above 11Mpps (received from one queue) with CRCSTRIP enabled, whyle 64+4-byte packets reach 14.2 Mpps (the theoretical maximum). Leaving the CRC in gets us 14.88Mpps for 60+4 byte frames, (and penalizes 64+4). The min-size case is important not just because it looks good in benchmarks, but also because this is the size of pure acks. Note we cannot leave CRCSTRIP on by default because it is incompatible with some other features (LRO etc.) Modified: head/sys/dev/ixgbe/ixgbe.c Modified: head/sys/dev/ixgbe/ixgbe.c ============================================================================== --- head/sys/dev/ixgbe/ixgbe.c Thu Jan 19 02:49:21 2012 (r230328) +++ head/sys/dev/ixgbe/ixgbe.c Thu Jan 19 09:36:19 2012 (r230329) @@ -1138,6 +1138,31 @@ ixgbe_init_locked(struct adapter *adapte msec_delay(1); } wmb(); +#ifdef DEV_NETMAP + /* + * In netmap mode, we must preserve the buffers made + * available to userspace before the if_init() + * (this is true by default on the TX side, because + * init makes all buffers available to userspace). + * + * netmap_reset() and the device specific routines + * (e.g. ixgbe_setup_receive_rings()) map these + * buffers at the end of the NIC ring, so here we + * must set the RDT (tail) register to make sure + * they are not overwritten. + * + * In this driver the NIC ring starts at RDH = 0, + * RDT points to the last slot available for reception (?), + * so RDT = num_rx_desc - 1 means the whole ring is available. + */ + if (ifp->if_capenable & IFCAP_NETMAP) { + struct netmap_adapter *na = NA(adapter->ifp); + struct netmap_kring *kring = &na->rx_rings[i]; + int t = na->num_rx_desc - 1 - kring->nr_hwavail; + + IXGBE_WRITE_REG(hw, IXGBE_RDT(i), t); + } else +#endif /* DEV_NETMAP */ IXGBE_WRITE_REG(hw, IXGBE_RDT(i), adapter->num_rx_desc - 1); } @@ -3903,6 +3928,21 @@ skip_head: lro->ifp = adapter->ifp; } +#ifdef DEV_NETMAP1 /* XXX experimental CRC strip */ + { + struct ixgbe_hw *hw = &adapter->hw; + u32 rdrxctl; + + rdrxctl = IXGBE_READ_REG(hw, IXGBE_RDRXCTL); + rdrxctl &= ~IXGBE_RDRXCTL_RSCFRSTSIZE; + if (slot) + rdrxctl &= ~IXGBE_RDRXCTL_CRCSTRIP; + else + rdrxctl |= IXGBE_RDRXCTL_CRCSTRIP; + rdrxctl |= IXGBE_RDRXCTL_RSCACKC; + IXGBE_WRITE_REG(hw, IXGBE_RDRXCTL, rdrxctl); + } +#endif /* DEV_NETMAP1 */ IXGBE_RX_UNLOCK(rxr); return (0); @@ -3982,6 +4022,12 @@ ixgbe_initialize_receive_units(struct ad hlreg |= IXGBE_HLREG0_JUMBOEN; else hlreg &= ~IXGBE_HLREG0_JUMBOEN; +#ifdef DEV_NETMAP1 /* XXX experimental CRCSTRIP */ + if (ifp->if_capenable & IFCAP_NETMAP) + hlreg &= ~IXGBE_HLREG0_RXCRCSTRP; + else + hlreg |= IXGBE_HLREG0_RXCRCSTRP; +#endif /* DEV_NETMAP1 */ IXGBE_WRITE_REG(hw, IXGBE_HLREG0, hlreg); bufsz = (adapter->rx_mbuf_sz + BSIZEPKT_ROUNDUP) >> IXGBE_SRRCTL_BSIZEPKT_SHIFT; @@ -4013,35 +4059,6 @@ ixgbe_initialize_receive_units(struct ad /* Setup the HW Rx Head and Tail Descriptor Pointers */ IXGBE_WRITE_REG(hw, IXGBE_RDH(i), 0); -#ifdef DEV_NETMAP - /* - * In netmap mode, we must preserve the buffers made - * available to userspace before the if_init() - * (this is true by default on the TX side, because - * init makes all buffers available to userspace). - * - * netmap_reset() and the device specific routines - * (e.g. ixgbe_setup_receive_rings()) map these - * buffers at the end of the NIC ring, so here we - * must set the RDT (tail) register to make sure - * they are not overwritten. - * - * In this driver the NIC ring starts at RDH = 0, - * RDT points to the first 'busy' slot, so RDT = 0 - * means the whole ring is available, and - * RDT = (num_rx_desc - X) means X slots are available. - * Computations are done modulo the ring size. - */ - if (ifp->if_capenable & IFCAP_NETMAP) { - struct netmap_adapter *na = NA(adapter->ifp); - struct netmap_kring *kring = &na->rx_rings[i]; - int t = na->num_rx_desc - kring->nr_hwavail; - - if (t >= na->num_rx_desc) - t -= adapter->num_rx_desc; - IXGBE_WRITE_REG(hw, IXGBE_RDT(i), t); - } else -#endif /* DEV_NETMAP */ IXGBE_WRITE_REG(hw, IXGBE_RDT(i), 0); }