From owner-svn-src-all@FreeBSD.ORG Thu Aug 16 22:33:56 2012 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C5CD6106564A; Thu, 16 Aug 2012 22:33:56 +0000 (UTC) (envelope-from np@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id AFFAF8FC0A; Thu, 16 Aug 2012 22:33:56 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.4/8.14.4) with ESMTP id q7GMXuRw079064; Thu, 16 Aug 2012 22:33:56 GMT (envelope-from np@svn.freebsd.org) Received: (from np@localhost) by svn.freebsd.org (8.14.4/8.14.4/Submit) id q7GMXuvd079060; Thu, 16 Aug 2012 22:33:56 GMT (envelope-from np@svn.freebsd.org) Message-Id: <201208162233.q7GMXuvd079060@svn.freebsd.org> From: Navdeep Parhar Date: Thu, 16 Aug 2012 22:33:56 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r239341 - head/sys/dev/cxgbe X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Aug 2012 22:33:56 -0000 Author: np Date: Thu Aug 16 22:33:56 2012 New Revision: 239341 URL: http://svn.freebsd.org/changeset/base/239341 Log: Initialize various DDP parameters in the main cxgbe(4) driver: - Setup multiple DDP page sizes. When the driver attempts DDP it will try to combine physically contiguous pages into regions of these sizes. - Set the indicate size such that the payload carried in the indicate can be copied in the header mbuf (and the 16K rx buffer can be recycled). - Set DDP threshold to the max payload that the chip will coalesce and deliver to the driver (this is ~16K by default, which is also why the offload rx queue is backed by 16K buffers). If the chip is able to coalesce up to the max it's allowed to, it's a good sign that the peer is transmitting in bulk without any TCP PSH. MFC after: 2 weeks Modified: head/sys/dev/cxgbe/adapter.h head/sys/dev/cxgbe/t4_main.c head/sys/dev/cxgbe/t4_sge.c Modified: head/sys/dev/cxgbe/adapter.h ============================================================================== --- head/sys/dev/cxgbe/adapter.h Thu Aug 16 20:54:52 2012 (r239340) +++ head/sys/dev/cxgbe/adapter.h Thu Aug 16 22:33:56 2012 (r239341) @@ -135,6 +135,7 @@ enum { #else FL_BUF_SIZES = 3, /* cluster, jumbo9k, jumbo16k */ #endif + OFLD_BUF_SIZE = MJUM16BYTES, /* size of fl buffer for TOE rxq */ CTRL_EQ_QSIZE = 128, @@ -143,6 +144,12 @@ enum { TX_WR_FLITS = SGE_MAX_WR_LEN / 8 }; +#ifdef T4_PKT_TIMESTAMP +#define RX_COPY_THRESHOLD (MINCLSIZE - 8) +#else +#define RX_COPY_THRESHOLD MINCLSIZE +#endif + enum { /* adapter intr_type */ INTR_INTX = (1 << 0), Modified: head/sys/dev/cxgbe/t4_main.c ============================================================================== --- head/sys/dev/cxgbe/t4_main.c Thu Aug 16 20:54:52 2012 (r239340) +++ head/sys/dev/cxgbe/t4_main.c Thu Aug 16 22:33:56 2012 (r239341) @@ -515,12 +515,16 @@ t4_attach(device_t dev) goto done; /* error message displayed already */ if (sc->flags & MASTER_PF) { + uint16_t indsz = min(RX_COPY_THRESHOLD - 1, M_INDICATESIZE); /* final tweaks to some settings */ t4_load_mtus(sc, sc->params.mtus, sc->params.a_wnd, sc->params.b_wnd); - t4_write_reg(sc, A_ULP_RX_TDDP_PSZ, V_HPZ0(PAGE_SHIFT - 12)); + /* 4K, 16K, 64K, 256K DDP "page sizes" */ + t4_write_reg(sc, A_ULP_RX_TDDP_PSZ, V_HPZ0(0) | V_HPZ1(2) | + V_HPZ2(4) | V_HPZ3(6)); + t4_set_reg_field(sc, A_ULP_RX_CTL, F_TDDPTAGTCB, F_TDDPTAGTCB); t4_set_reg_field(sc, A_TP_PARA_REG3, F_TUNNELCNGDROP0 | F_TUNNELCNGDROP1 | F_TUNNELCNGDROP2 | F_TUNNELCNGDROP3, F_TUNNELCNGDROP0 | F_TUNNELCNGDROP1 | F_TUNNELCNGDROP2 | @@ -528,7 +532,7 @@ t4_attach(device_t dev) t4_set_reg_field(sc, A_TP_PARA_REG5, V_INDICATESIZE(M_INDICATESIZE) | F_REARMDDPOFFSET | F_RESETDDPOFFSET, - V_INDICATESIZE(M_INDICATESIZE) | + V_INDICATESIZE(indsz) | F_REARMDDPOFFSET | F_RESETDDPOFFSET); } else { /* @@ -3228,10 +3232,13 @@ t4_sysctls(struct adapter *sc) sc->tt.ddp = 0; SYSCTL_ADD_INT(ctx, children, OID_AUTO, "ddp", CTLFLAG_RW, &sc->tt.ddp, 0, "DDP allowed"); - sc->tt.indsz = M_INDICATESIZE; + + sc->tt.indsz = G_INDICATESIZE(t4_read_reg(sc, A_TP_PARA_REG5)); SYSCTL_ADD_INT(ctx, children, OID_AUTO, "indsz", CTLFLAG_RW, &sc->tt.indsz, 0, "DDP max indicate size allowed"); - sc->tt.ddp_thres = 3*4096; + + sc->tt.ddp_thres = + G_RXCOALESCESIZE(t4_read_reg(sc, A_TP_PARA_REG2)); SYSCTL_ADD_INT(ctx, children, OID_AUTO, "ddp_thres", CTLFLAG_RW, &sc->tt.ddp_thres, 0, "DDP threshold"); } Modified: head/sys/dev/cxgbe/t4_sge.c ============================================================================== --- head/sys/dev/cxgbe/t4_sge.c Thu Aug 16 20:54:52 2012 (r239340) +++ head/sys/dev/cxgbe/t4_sge.c Thu Aug 16 22:33:56 2012 (r239341) @@ -627,7 +627,7 @@ t4_setup_port_queues(struct port_info *p snprintf(name, sizeof(name), "%s ofld_rxq%d-fl", device_get_nameunit(pi->dev), i); - init_fl(&ofld_rxq->fl, pi->qsize_rxq / 8, MJUM16BYTES, name); + init_fl(&ofld_rxq->fl, pi->qsize_rxq / 8, OFLD_BUF_SIZE, name); if (sc->flags & INTR_DIRECT || (sc->intr_count > 1 && pi->nofldrxq > pi->nrxq)) { @@ -1022,13 +1022,6 @@ service_iq(struct sge_iq *iq, int budget return (0); } - -#ifdef T4_PKT_TIMESTAMP -#define RX_COPY_THRESHOLD (MINCLSIZE - 8) -#else -#define RX_COPY_THRESHOLD MINCLSIZE -#endif - static struct mbuf * get_fl_payload(struct adapter *sc, struct sge_fl *fl, uint32_t len_newbuf, int *fl_bufs_used)