Date: Thu, 19 Jan 2012 09:36:19 +0000 (UTC) From: Luigi Rizzo <luigi@FreeBSD.org> To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r230329 - head/sys/dev/ixgbe Message-ID: <201201190936.q0J9aJX8080855@svn.freebsd.org>
next in thread | raw e-mail | index | archive | help
Author: luigi Date: Thu Jan 19 09:36:19 2012 New Revision: 230329 URL: http://svn.freebsd.org/changeset/base/230329 Log: netmap-related changes: 1. correct the initialization of RDT when there is an ixgbe_init() while a netmap client is active. This code was previously in ixgbe_initialize_receive_units() but RDT is overwritten shortly afterwards in ixgbe_init_locked() 2. add code (not active yet) to disable CRCSTRIP while in netmap mode. From all evidence i could gather, it seems that when the 82599 has to write a data block that is not a full cache line, it first reads the line (64 bytes) and then writes back the updated version. This hurts reception of min-sized frames, which are only 60 bytes if the CRC is stripped: i could never get above 11Mpps (received from one queue) with CRCSTRIP enabled, whyle 64+4-byte packets reach 14.2 Mpps (the theoretical maximum). Leaving the CRC in gets us 14.88Mpps for 60+4 byte frames, (and penalizes 64+4). The min-size case is important not just because it looks good in benchmarks, but also because this is the size of pure acks. Note we cannot leave CRCSTRIP on by default because it is incompatible with some other features (LRO etc.) Modified: head/sys/dev/ixgbe/ixgbe.c Modified: head/sys/dev/ixgbe/ixgbe.c ============================================================================== --- head/sys/dev/ixgbe/ixgbe.c Thu Jan 19 02:49:21 2012 (r230328) +++ head/sys/dev/ixgbe/ixgbe.c Thu Jan 19 09:36:19 2012 (r230329) @@ -1138,6 +1138,31 @@ ixgbe_init_locked(struct adapter *adapte msec_delay(1); } wmb(); +#ifdef DEV_NETMAP + /* + * In netmap mode, we must preserve the buffers made + * available to userspace before the if_init() + * (this is true by default on the TX side, because + * init makes all buffers available to userspace). + * + * netmap_reset() and the device specific routines + * (e.g. ixgbe_setup_receive_rings()) map these + * buffers at the end of the NIC ring, so here we + * must set the RDT (tail) register to make sure + * they are not overwritten. + * + * In this driver the NIC ring starts at RDH = 0, + * RDT points to the last slot available for reception (?), + * so RDT = num_rx_desc - 1 means the whole ring is available. + */ + if (ifp->if_capenable & IFCAP_NETMAP) { + struct netmap_adapter *na = NA(adapter->ifp); + struct netmap_kring *kring = &na->rx_rings[i]; + int t = na->num_rx_desc - 1 - kring->nr_hwavail; + + IXGBE_WRITE_REG(hw, IXGBE_RDT(i), t); + } else +#endif /* DEV_NETMAP */ IXGBE_WRITE_REG(hw, IXGBE_RDT(i), adapter->num_rx_desc - 1); } @@ -3903,6 +3928,21 @@ skip_head: lro->ifp = adapter->ifp; } +#ifdef DEV_NETMAP1 /* XXX experimental CRC strip */ + { + struct ixgbe_hw *hw = &adapter->hw; + u32 rdrxctl; + + rdrxctl = IXGBE_READ_REG(hw, IXGBE_RDRXCTL); + rdrxctl &= ~IXGBE_RDRXCTL_RSCFRSTSIZE; + if (slot) + rdrxctl &= ~IXGBE_RDRXCTL_CRCSTRIP; + else + rdrxctl |= IXGBE_RDRXCTL_CRCSTRIP; + rdrxctl |= IXGBE_RDRXCTL_RSCACKC; + IXGBE_WRITE_REG(hw, IXGBE_RDRXCTL, rdrxctl); + } +#endif /* DEV_NETMAP1 */ IXGBE_RX_UNLOCK(rxr); return (0); @@ -3982,6 +4022,12 @@ ixgbe_initialize_receive_units(struct ad hlreg |= IXGBE_HLREG0_JUMBOEN; else hlreg &= ~IXGBE_HLREG0_JUMBOEN; +#ifdef DEV_NETMAP1 /* XXX experimental CRCSTRIP */ + if (ifp->if_capenable & IFCAP_NETMAP) + hlreg &= ~IXGBE_HLREG0_RXCRCSTRP; + else + hlreg |= IXGBE_HLREG0_RXCRCSTRP; +#endif /* DEV_NETMAP1 */ IXGBE_WRITE_REG(hw, IXGBE_HLREG0, hlreg); bufsz = (adapter->rx_mbuf_sz + BSIZEPKT_ROUNDUP) >> IXGBE_SRRCTL_BSIZEPKT_SHIFT; @@ -4013,35 +4059,6 @@ ixgbe_initialize_receive_units(struct ad /* Setup the HW Rx Head and Tail Descriptor Pointers */ IXGBE_WRITE_REG(hw, IXGBE_RDH(i), 0); -#ifdef DEV_NETMAP - /* - * In netmap mode, we must preserve the buffers made - * available to userspace before the if_init() - * (this is true by default on the TX side, because - * init makes all buffers available to userspace). - * - * netmap_reset() and the device specific routines - * (e.g. ixgbe_setup_receive_rings()) map these - * buffers at the end of the NIC ring, so here we - * must set the RDT (tail) register to make sure - * they are not overwritten. - * - * In this driver the NIC ring starts at RDH = 0, - * RDT points to the first 'busy' slot, so RDT = 0 - * means the whole ring is available, and - * RDT = (num_rx_desc - X) means X slots are available. - * Computations are done modulo the ring size. - */ - if (ifp->if_capenable & IFCAP_NETMAP) { - struct netmap_adapter *na = NA(adapter->ifp); - struct netmap_kring *kring = &na->rx_rings[i]; - int t = na->num_rx_desc - kring->nr_hwavail; - - if (t >= na->num_rx_desc) - t -= adapter->num_rx_desc; - IXGBE_WRITE_REG(hw, IXGBE_RDT(i), t); - } else -#endif /* DEV_NETMAP */ IXGBE_WRITE_REG(hw, IXGBE_RDT(i), 0); }
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201201190936.q0J9aJX8080855>