From owner-svn-src-all@FreeBSD.ORG Fri Apr 2 17:50:53 2010 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 369061065702; Fri, 2 Apr 2010 17:50:53 +0000 (UTC) (envelope-from np@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 1BAD38FC15; Fri, 2 Apr 2010 17:50:53 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id o32HorEP028995; Fri, 2 Apr 2010 17:50:53 GMT (envelope-from np@svn.freebsd.org) Received: (from np@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id o32Hoq43028991; Fri, 2 Apr 2010 17:50:52 GMT (envelope-from np@svn.freebsd.org) Message-Id: <201004021750.o32Hoq43028991@svn.freebsd.org> From: Navdeep Parhar Date: Fri, 2 Apr 2010 17:50:52 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r206109 - head/sys/dev/cxgb X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Apr 2010 17:50:53 -0000 Author: np Date: Fri Apr 2 17:50:52 2010 New Revision: 206109 URL: http://svn.freebsd.org/changeset/base/206109 Log: Increase response queue size to avoid starvation, add a counter to track it when it does occur. Modified: head/sys/dev/cxgb/cxgb_adapter.h head/sys/dev/cxgb/cxgb_main.c head/sys/dev/cxgb/cxgb_sge.c Modified: head/sys/dev/cxgb/cxgb_adapter.h ============================================================================== --- head/sys/dev/cxgb/cxgb_adapter.h Fri Apr 2 17:48:01 2010 (r206108) +++ head/sys/dev/cxgb/cxgb_adapter.h Fri Apr 2 17:50:52 2010 (r206109) @@ -139,7 +139,7 @@ enum { #define FL_Q_SIZE 4096 #define JUMBO_Q_SIZE 1024 -#define RSPQ_Q_SIZE 1024 +#define RSPQ_Q_SIZE 2048 #define TX_ETH_Q_SIZE 1024 #define TX_OFLD_Q_SIZE 1024 #define TX_CTRL_Q_SIZE 256 @@ -179,6 +179,7 @@ struct sge_rspq { uint32_t offload_bundles; uint32_t pure_rsps; uint32_t unhandled_irqs; + uint32_t starved; bus_addr_t phys_addr; bus_dma_tag_t desc_tag; Modified: head/sys/dev/cxgb/cxgb_main.c ============================================================================== --- head/sys/dev/cxgb/cxgb_main.c Fri Apr 2 17:48:01 2010 (r206108) +++ head/sys/dev/cxgb/cxgb_main.c Fri Apr 2 17:50:52 2010 (r206109) @@ -2398,25 +2398,33 @@ cxgb_tick_handler(void *arg, int count) if (p->rev == T3_REV_B2 && p->nports < 4 && sc->open_device_map) check_t3b2_mac(sc); - cause = t3_read_reg(sc, A_SG_INT_CAUSE); - reset = 0; - if (cause & F_FLEMPTY) { + cause = t3_read_reg(sc, A_SG_INT_CAUSE) & (F_RSPQSTARVE | F_FLEMPTY); + if (cause) { struct sge_qset *qs = &sc->sge.qs[0]; + uint32_t mask, v; - i = 0; - reset |= F_FLEMPTY; + v = t3_read_reg(sc, A_SG_RSPQ_FL_STATUS) & ~0xff00; - cause = (t3_read_reg(sc, A_SG_RSPQ_FL_STATUS) >> - S_FL0EMPTY) & 0xffff; - while (cause) { - qs->fl[i].empty += (cause & 1); - if (i) - qs++; - i ^= 1; - cause >>= 1; + mask = 1; + for (i = 0; i < SGE_QSETS; i++) { + if (v & mask) + qs[i].rspq.starved++; + mask <<= 1; } + + mask <<= SGE_QSETS; /* skip RSPQXDISABLED */ + + for (i = 0; i < SGE_QSETS * 2; i++) { + if (v & mask) { + qs[i / 2].fl[i % 2].empty++; + } + mask <<= 1; + } + + /* clear */ + t3_write_reg(sc, A_SG_RSPQ_FL_STATUS, v); + t3_write_reg(sc, A_SG_INT_CAUSE, cause); } - t3_write_reg(sc, A_SG_INT_CAUSE, reset); for (i = 0; i < sc->params.nports; i++) { struct port_info *pi = &sc->port[i]; Modified: head/sys/dev/cxgb/cxgb_sge.c ============================================================================== --- head/sys/dev/cxgb/cxgb_sge.c Fri Apr 2 17:48:01 2010 (r206108) +++ head/sys/dev/cxgb/cxgb_sge.c Fri Apr 2 17:50:52 2010 (r206109) @@ -3586,6 +3586,9 @@ t3_add_configured_sysctls(adapter_t *sc) SYSCTL_ADD_UINT(ctx, rspqpoidlist, OID_AUTO, "credits", CTLFLAG_RD, &qs->rspq.credits, 0, "#credits"); + SYSCTL_ADD_UINT(ctx, rspqpoidlist, OID_AUTO, "starved", + CTLFLAG_RD, &qs->rspq.starved, + 0, "#times starved"); SYSCTL_ADD_XLONG(ctx, rspqpoidlist, OID_AUTO, "phys_addr", CTLFLAG_RD, &qs->rspq.phys_addr, "physical_address_of the queue");