From owner-svn-src-head@FreeBSD.ORG Fri Sep 12 05:25:57 2014 Return-Path: Delivered-To: svn-src-head@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 543EF926; Fri, 12 Sep 2014 05:25:57 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 332319DC; Fri, 12 Sep 2014 05:25:57 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.9/8.14.9) with ESMTP id s8C5Pvv6092804; Fri, 12 Sep 2014 05:25:57 GMT (envelope-from np@FreeBSD.org) Received: (from np@localhost) by svn.freebsd.org (8.14.9/8.14.9/Submit) id s8C5PuJl092801; Fri, 12 Sep 2014 05:25:56 GMT (envelope-from np@FreeBSD.org) Message-Id: <201409120525.s8C5PuJl092801@svn.freebsd.org> X-Authentication-Warning: svn.freebsd.org: np set sender to np@FreeBSD.org using -f From: Navdeep Parhar Date: Fri, 12 Sep 2014 05:25:56 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r271450 - in head: share/man/man4 sys/dev/cxgbe X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Sep 2014 05:25:57 -0000 Author: np Date: Fri Sep 12 05:25:56 2014 New Revision: 271450 URL: http://svnweb.freebsd.org/changeset/base/271450 Log: cxgbe(4): knobs to enable/disable PAUSE frame based flow control. MFC after: 1 week Modified: head/share/man/man4/cxgbe.4 head/sys/dev/cxgbe/t4_main.c Modified: head/share/man/man4/cxgbe.4 ============================================================================== --- head/share/man/man4/cxgbe.4 Fri Sep 12 04:15:35 2014 (r271449) +++ head/share/man/man4/cxgbe.4 Fri Sep 12 05:25:56 2014 (r271450) @@ -240,8 +240,19 @@ The default is -1 which lets the driver Controls the hardware response to congestion. -1 disables congestion feedback and is not recommended. 0 instructs the hardware to backpressure its pipeline on congestion. -This usually results in the port emitting pause frames. +This usually results in the port emitting PAUSE frames. 1 instructs the hardware to drop frames destined for congested queues. +.It Va hw.cxgbe.pause_settings +PAUSE frame settings. +Bit 0 is rx_pause, bit 1 is tx_pause. +rx_pause = 1 instructs the hardware to heed incoming PAUSE frames, 0 instructs +it to ignore them. +tx_pause = 1 allows the hardware to emit PAUSE frames when its receive FIFO +reaches a high threshold, 0 prohibits the hardware from emitting PAUSE frames. +The default is 3 (both rx_pause and tx_pause = 1). +This tunable establishes the default PAUSE settings for all ports. +Settings can be displayed and controlled on a per-port basis via the +dev.cxgbe.X.pause_settings (dev.cxl.X.pause_settings for T5 cards) sysctl. .It Va hw.cxgbe.buffer_packing Allow the hardware to deliver multiple frames in the same receive buffer opportunistically. Modified: head/sys/dev/cxgbe/t4_main.c ============================================================================== --- head/sys/dev/cxgbe/t4_main.c Fri Sep 12 04:15:35 2014 (r271449) +++ head/sys/dev/cxgbe/t4_main.c Fri Sep 12 05:25:56 2014 (r271450) @@ -281,6 +281,15 @@ static char t4_cfg_file[32] = DEFAULT_CF TUNABLE_STR("hw.cxgbe.config_file", t4_cfg_file, sizeof(t4_cfg_file)); /* + * PAUSE settings (bit 0, 1 = rx_pause, tx_pause respectively). + * rx_pause = 1 to heed incoming PAUSE frames, 0 to ignore them. + * tx_pause = 1 to emit PAUSE frames when the rx FIFO reaches its high water + * mark or when signalled to do so, 0 to never emit PAUSE. + */ +static int t4_pause_settings = PAUSE_TX | PAUSE_RX; +TUNABLE_INT("hw.cxgbe.pause_settings", &t4_pause_settings); + +/* * Firmware auto-install by driver during attach (0, 1, 2 = prohibited, allowed, * encouraged respectively). */ @@ -393,6 +402,7 @@ static int sysctl_holdoff_tmr_idx(SYSCTL static int sysctl_holdoff_pktc_idx(SYSCTL_HANDLER_ARGS); static int sysctl_qsize_rxq(SYSCTL_HANDLER_ARGS); static int sysctl_qsize_txq(SYSCTL_HANDLER_ARGS); +static int sysctl_pause_settings(SYSCTL_HANDLER_ARGS); static int sysctl_handle_t4_reg64(SYSCTL_HANDLER_ARGS); static int sysctl_temperature(SYSCTL_HANDLER_ARGS); #ifdef SBUF_DRAIN @@ -697,6 +707,12 @@ t4_attach(device_t dev) sc->port[i] = NULL; goto done; } + + pi->link_cfg.requested_fc &= ~(PAUSE_TX | PAUSE_RX); + pi->link_cfg.requested_fc |= t4_pause_settings; + pi->link_cfg.fc &= ~(PAUSE_TX | PAUSE_RX); + pi->link_cfg.fc |= t4_pause_settings; + rc = -t4_link_start(sc, sc->mbox, pi->tx_chan, &pi->link_cfg); if (rc != 0) { device_printf(dev, "port %d l1cfg failed: %d\n", i, rc); @@ -4771,6 +4787,10 @@ cxgbe_sysctls(struct port_info *pi) CTLTYPE_INT | CTLFLAG_RW, pi, 0, sysctl_qsize_txq, "I", "tx queue size"); + SYSCTL_ADD_PROC(ctx, children, OID_AUTO, "pause_settings", + CTLTYPE_STRING | CTLFLAG_RW, pi, PAUSE_TX, sysctl_pause_settings, + "A", "PAUSE settings (bit 0 = rx_pause, bit 1 = tx_pause)"); + /* * dev.cxgbe.X.stats. */ @@ -5152,6 +5172,65 @@ sysctl_qsize_txq(SYSCTL_HANDLER_ARGS) } static int +sysctl_pause_settings(SYSCTL_HANDLER_ARGS) +{ + struct port_info *pi = arg1; + struct adapter *sc = pi->adapter; + struct link_config *lc = &pi->link_cfg; + int rc; + + if (req->newptr == NULL) { + struct sbuf *sb; + static char *bits = "\20\1PAUSE_RX\2PAUSE_TX"; + + rc = sysctl_wire_old_buffer(req, 0); + if (rc != 0) + return(rc); + + sb = sbuf_new_for_sysctl(NULL, NULL, 128, req); + if (sb == NULL) + return (ENOMEM); + + sbuf_printf(sb, "%b", lc->fc & (PAUSE_TX | PAUSE_RX), bits); + rc = sbuf_finish(sb); + sbuf_delete(sb); + } else { + char s[2]; + int n; + + s[0] = '0' + (lc->requested_fc & (PAUSE_TX | PAUSE_RX)); + s[1] = 0; + + rc = sysctl_handle_string(oidp, s, sizeof(s), req); + if (rc != 0) + return(rc); + + if (s[1] != 0) + return (EINVAL); + if (s[0] < '0' || s[0] > '9') + return (EINVAL); /* not a number */ + n = s[0] - '0'; + if (n & ~(PAUSE_TX | PAUSE_RX)) + return (EINVAL); /* some other bit is set too */ + + rc = begin_synchronized_op(sc, pi, SLEEP_OK | INTR_OK, "t4PAUSE"); + if (rc) + return (rc); + if ((lc->requested_fc & (PAUSE_TX | PAUSE_RX)) != n) { + int link_ok = lc->link_ok; + + lc->requested_fc &= ~(PAUSE_TX | PAUSE_RX); + lc->requested_fc |= n; + rc = -t4_link_start(sc, sc->mbox, pi->tx_chan, lc); + lc->link_ok = link_ok; /* restore */ + } + end_synchronized_op(sc, 0); + } + + return (rc); +} + +static int sysctl_handle_t4_reg64(SYSCTL_HANDLER_ARGS) { struct adapter *sc = arg1;