From owner-svn-src-head@freebsd.org Fri Dec 30 08:59:51 2016 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9ADABC97D7E; Fri, 30 Dec 2016 08:59:51 +0000 (UTC) (envelope-from np@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5C1F81B4E; Fri, 30 Dec 2016 08:59:51 +0000 (UTC) (envelope-from np@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id uBU8xoB0091084; Fri, 30 Dec 2016 08:59:50 GMT (envelope-from np@FreeBSD.org) Received: (from np@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id uBU8xoBd091077; Fri, 30 Dec 2016 08:59:50 GMT (envelope-from np@FreeBSD.org) Message-Id: <201612300859.uBU8xoBd091077@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: np set sender to np@FreeBSD.org using -f From: Navdeep Parhar Date: Fri, 30 Dec 2016 08:59:50 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r310807 - in head: share/man/man4 sys/dev/cxgbe sys/dev/cxgbe/common X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Dec 2016 08:59:51 -0000 Author: np Date: Fri Dec 30 08:59:49 2016 New Revision: 310807 URL: https://svnweb.freebsd.org/changeset/base/310807 Log: cxgbe(4): Updates to link configuration. - Update struct link_settings and associated shared code. - Add tunables to control FEC and autonegotiation. All ports inherit these values as their initial settings. hw.cxgbe.fec hw.cxgbe.autoneg - Add per-port sysctls to control FEC and autonegotiation. These can be modified at any time. dev...fec dev...autoneg MFC after: 3 days Sponsored by: Chelsio Communications Modified: head/share/man/man4/cxgbe.4 head/sys/dev/cxgbe/adapter.h head/sys/dev/cxgbe/common/common.h head/sys/dev/cxgbe/common/t4_hw.c head/sys/dev/cxgbe/t4_main.c head/sys/dev/cxgbe/t4_vf.c Modified: head/share/man/man4/cxgbe.4 ============================================================================== --- head/share/man/man4/cxgbe.4 Fri Dec 30 08:37:30 2016 (r310806) +++ head/share/man/man4/cxgbe.4 Fri Dec 30 08:59:49 2016 (r310807) @@ -289,6 +289,24 @@ The default is 3 (both rx_pause and tx_p This tunable establishes the default PAUSE settings for all ports. Settings can be displayed and controlled on a per-port basis via the dev..X.pause_settings sysctl. +.It Va hw.cxgbe.fec +FEC (Forward Error Correction) settings. +0 diables FEC. +Bit 0 enables RS FEC, bit 1 enables BASE-R RS, bit 3 is reserved. +The default is -1 which lets the driver pick a value. +This tunable establishes the default FEC settings for all ports. +Settings can be displayed and controlled on a per-port basis via the +dev..X.fec sysctl. +.It Va hw.cxgbe.autoneg +Link autonegotiation settings. +This tunable establishes the default autonegotiation settings for all ports. +Settings can be displayed and controlled on a per-port basis via the +dev..X.autoneg sysctl. +0 disables autonegotiation. +1 enables autonegotiation. +The default is -1 which lets the driver pick a value. +dev..X.autoneg is -1 for port and module combinations that do not support +autonegotiation. .It Va hw.cxgbe.buffer_packing Allow the hardware to deliver multiple frames in the same receive buffer opportunistically. Modified: head/sys/dev/cxgbe/adapter.h ============================================================================== --- head/sys/dev/cxgbe/adapter.h Fri Dec 30 08:37:30 2016 (r310806) +++ head/sys/dev/cxgbe/adapter.h Fri Dec 30 08:59:49 2016 (r310807) @@ -265,7 +265,6 @@ struct port_info { uint8_t tx_chan; uint8_t rx_chan_map; /* rx MPS channel bitmap */ - int linkdnrc; struct link_config link_cfg; struct timeval last_refreshed; @@ -1098,7 +1097,7 @@ int t4_os_find_pci_capability(struct ada int t4_os_pci_save_state(struct adapter *); int t4_os_pci_restore_state(struct adapter *); void t4_os_portmod_changed(const struct adapter *, int); -void t4_os_link_changed(struct adapter *, int, int, int); +void t4_os_link_changed(struct adapter *, int, int); void t4_iterate(void (*)(struct adapter *, void *), void *); void t4_init_devnames(struct adapter *); void t4_add_adapter(struct adapter *); Modified: head/sys/dev/cxgbe/common/common.h ============================================================================== --- head/sys/dev/cxgbe/common/common.h Fri Dec 30 08:37:30 2016 (r310806) +++ head/sys/dev/cxgbe/common/common.h Fri Dec 30 08:59:49 2016 (r310807) @@ -62,6 +62,12 @@ enum { PAUSE_AUTONEG = 1 << 2 }; +enum { + FEC_RS = 1 << 0, + FEC_BASER_RS = 1 << 1, + FEC_RESERVED = 1 << 2, +}; + struct port_stats { u64 tx_octets; /* total # of octets in good frames */ u64 tx_frames; /* all good frames */ @@ -392,12 +398,16 @@ struct trace_params { struct link_config { unsigned short supported; /* link capabilities */ unsigned short advertising; /* advertised capabilities */ - unsigned int requested_speed; /* speed user has requested */ - unsigned int speed; /* actual link speed */ + unsigned short lp_advertising; /* peer advertised capabilities */ + unsigned int requested_speed; /* speed user has requested */ + unsigned int speed; /* actual link speed */ unsigned char requested_fc; /* flow control user has requested */ unsigned char fc; /* actual link flow control */ + unsigned char requested_fec; /* FEC user has requested */ + unsigned char fec; /* actual FEC */ unsigned char autoneg; /* autonegotiating? */ unsigned char link_ok; /* link up? */ + unsigned char link_down_rc; /* link down reason */ }; #include "adapter.h" Modified: head/sys/dev/cxgbe/common/t4_hw.c ============================================================================== --- head/sys/dev/cxgbe/common/t4_hw.c Fri Dec 30 08:37:30 2016 (r310806) +++ head/sys/dev/cxgbe/common/t4_hw.c Fri Dec 30 08:59:49 2016 (r310807) @@ -3683,9 +3683,7 @@ void t4_ulprx_read_la(struct adapter *ad } } -#define ADVERT_MASK (FW_PORT_CAP_SPEED_100M | FW_PORT_CAP_SPEED_1G |\ - FW_PORT_CAP_SPEED_10G | FW_PORT_CAP_SPEED_25G | \ - FW_PORT_CAP_SPEED_40G | FW_PORT_CAP_SPEED_100G | \ +#define ADVERT_MASK (V_FW_PORT_CAP_SPEED(M_FW_PORT_CAP_SPEED) | \ FW_PORT_CAP_ANEG) /** @@ -3705,14 +3703,23 @@ int t4_link_l1cfg(struct adapter *adap, struct link_config *lc) { struct fw_port_cmd c; - unsigned int fc = 0, mdi = V_FW_PORT_CAP_MDI(FW_PORT_CAP_MDI_AUTO); + unsigned int mdi = V_FW_PORT_CAP_MDI(FW_PORT_CAP_MDI_AUTO); + unsigned int fc, fec; - lc->link_ok = 0; + fc = 0; if (lc->requested_fc & PAUSE_RX) fc |= FW_PORT_CAP_FC_RX; if (lc->requested_fc & PAUSE_TX) fc |= FW_PORT_CAP_FC_TX; + fec = 0; + if (lc->requested_fec & FEC_RS) + fec |= FW_PORT_CAP_FEC_RS; + if (lc->requested_fec & FEC_BASER_RS) + fec |= FW_PORT_CAP_FEC_BASER_RS; + if (lc->requested_fec & FEC_RESERVED) + fec |= FW_PORT_CAP_FEC_RESERVED; + memset(&c, 0, sizeof(c)); c.op_to_portid = cpu_to_be32(V_FW_CMD_OP(FW_PORT_CMD) | F_FW_CMD_REQUEST | F_FW_CMD_EXEC | @@ -3723,13 +3730,16 @@ int t4_link_l1cfg(struct adapter *adap, if (!(lc->supported & FW_PORT_CAP_ANEG)) { c.u.l1cfg.rcap = cpu_to_be32((lc->supported & ADVERT_MASK) | - fc); - lc->fc = lc->requested_fc & (PAUSE_RX | PAUSE_TX); + fc | fec); + lc->fc = lc->requested_fc & ~PAUSE_AUTONEG; + lc->fec = lc->requested_fec; } else if (lc->autoneg == AUTONEG_DISABLE) { - c.u.l1cfg.rcap = cpu_to_be32(lc->requested_speed | fc | mdi); - lc->fc = lc->requested_fc & (PAUSE_RX | PAUSE_TX); + c.u.l1cfg.rcap = cpu_to_be32(lc->requested_speed | + fc | fec | mdi); + lc->fc = lc->requested_fc & ~PAUSE_AUTONEG; + lc->fec = lc->requested_fec; } else - c.u.l1cfg.rcap = cpu_to_be32(lc->advertising | fc | mdi); + c.u.l1cfg.rcap = cpu_to_be32(lc->advertising | fc | fec | mdi); return t4_wr_mbox(adap, mbox, &c, sizeof(c), NULL); } @@ -7517,18 +7527,14 @@ int t4_handle_fw_rpl(struct adapter *ada } if (link_ok != lc->link_ok || speed != lc->speed || fc != lc->fc) { /* something changed */ - int reason; - if (!link_ok && lc->link_ok) - reason = G_FW_PORT_CMD_LINKDNRC(stat); - else - reason = -1; - + lc->link_down_rc = G_FW_PORT_CMD_LINKDNRC(stat); lc->link_ok = link_ok; lc->speed = speed; lc->fc = fc; lc->supported = be16_to_cpu(p->u.info.pcap); - t4_os_link_changed(adap, i, link_ok, reason); + lc->lp_advertising = be16_to_cpu(p->u.info.lpacap); + t4_os_link_changed(adap, i, link_ok); } } else { CH_WARN_RATELIMIT(adap, "Unknown firmware reply %d\n", opcode); @@ -7562,17 +7568,34 @@ static void get_pci_mode(struct adapter /** * init_link_config - initialize a link's SW state * @lc: structure holding the link state - * @caps: link capabilities + * @pcaps: supported link capabilities + * @acaps: advertised link capabilities * * Initializes the SW state maintained for each link, including the link's * capabilities and default speed/flow-control/autonegotiation settings. */ -static void init_link_config(struct link_config *lc, unsigned int caps) +static void init_link_config(struct link_config *lc, unsigned int pcaps, + unsigned int acaps) { - lc->supported = caps; + unsigned int fec; + + lc->supported = pcaps; + lc->lp_advertising = 0; lc->requested_speed = 0; lc->speed = 0; lc->requested_fc = lc->fc = PAUSE_RX | PAUSE_TX; + lc->link_ok = 0; + lc->link_down_rc = 255; + + fec = 0; + if (acaps & FW_PORT_CAP_FEC_RS) + fec |= FEC_RS; + if (acaps & FW_PORT_CAP_FEC_BASER_RS) + fec |= FEC_BASER_RS; + if (acaps & FW_PORT_CAP_FEC_RESERVED) + fec |= FEC_RESERVED; + lc->requested_fec = lc->fec = fec; + if (lc->supported & FW_PORT_CAP_ANEG) { lc->advertising = lc->supported & ADVERT_MASK; lc->autoneg = AUTONEG_ENABLE; @@ -8126,7 +8149,8 @@ int t4_port_init(struct adapter *adap, i p->port_type = G_FW_PORT_CMD_PTYPE(ret); p->mod_type = G_FW_PORT_CMD_MODTYPE(ret); - init_link_config(&p->link_cfg, be16_to_cpu(c.u.info.pcap)); + init_link_config(&p->link_cfg, be16_to_cpu(c.u.info.pcap), + be16_to_cpu(c.u.info.acap)); } ret = t4_alloc_vi(adap, mbox, j, pf, vf, 1, addr, &rss_size); Modified: head/sys/dev/cxgbe/t4_main.c ============================================================================== --- head/sys/dev/cxgbe/t4_main.c Fri Dec 30 08:37:30 2016 (r310806) +++ head/sys/dev/cxgbe/t4_main.c Fri Dec 30 08:59:49 2016 (r310807) @@ -359,6 +359,24 @@ static int t4_pause_settings = PAUSE_TX TUNABLE_INT("hw.cxgbe.pause_settings", &t4_pause_settings); /* + * Forward Error Correction settings (bit 0, 1, 2 = FEC_RS, FEC_BASER_RS, + * FEC_RESERVED respectively). + * -1 to run with the firmware default. + * 0 to disable FEC. + */ +static int t4_fec = -1; +TUNABLE_INT("hw.cxgbe.fec", &t4_fec); + +/* + * Link autonegotiation. + * -1 to run with the firmware default. + * 0 to disable. + * 1 to enable. + */ +static int t4_autoneg = -1; +TUNABLE_INT("hw.cxgbe.autoneg", &t4_autoneg); + +/* * Firmware auto-install by driver during attach (0, 1, 2 = prohibited, allowed, * encouraged respectively). */ @@ -493,6 +511,8 @@ static int sysctl_holdoff_pktc_idx(SYSCT static int sysctl_qsize_rxq(SYSCTL_HANDLER_ARGS); static int sysctl_qsize_txq(SYSCTL_HANDLER_ARGS); static int sysctl_pause_settings(SYSCTL_HANDLER_ARGS); +static int sysctl_fec(SYSCTL_HANDLER_ARGS); +static int sysctl_autoneg(SYSCTL_HANDLER_ARGS); static int sysctl_handle_t4_reg64(SYSCTL_HANDLER_ARGS); static int sysctl_temperature(SYSCTL_HANDLER_ARGS); #ifdef SBUF_DRAIN @@ -922,6 +942,7 @@ t4_attach(device_t dev) n10g = n1g = 0; for_each_port(sc, i) { struct port_info *pi; + struct link_config *lc; pi = malloc(sizeof(*pi), M_CXGBE, M_ZERO | M_WAITOK); sc->port[i] = pi; @@ -950,12 +971,19 @@ t4_attach(device_t dev) goto done; } - pi->link_cfg.requested_fc &= ~(PAUSE_TX | PAUSE_RX); - pi->link_cfg.requested_fc |= t4_pause_settings; - pi->link_cfg.fc &= ~(PAUSE_TX | PAUSE_RX); - pi->link_cfg.fc |= t4_pause_settings; + lc = &pi->link_cfg; + lc->requested_fc &= ~(PAUSE_TX | PAUSE_RX); + lc->requested_fc |= t4_pause_settings; + if (t4_fec != -1) { + lc->requested_fec = t4_fec & + G_FW_PORT_CAP_FEC(lc->supported); + } + if (lc->supported & FW_PORT_CAP_ANEG && t4_autoneg != -1) { + lc->autoneg = t4_autoneg ? AUTONEG_ENABLE : + AUTONEG_DISABLE; + } - rc = -t4_link_l1cfg(sc, sc->mbox, pi->tx_chan, &pi->link_cfg); + rc = -t4_link_l1cfg(sc, sc->mbox, pi->tx_chan, lc); if (rc != 0) { device_printf(dev, "port %d l1cfg failed: %d\n", i, rc); free(pi->vi, M_CXGBE); @@ -978,8 +1006,6 @@ t4_attach(device_t dev) n1g++; } - pi->linkdnrc = -1; - pi->dev = device_add_child(dev, sc->names->ifnet_name, -1); if (pi->dev == NULL) { device_printf(dev, @@ -4060,8 +4086,8 @@ cxgbe_uninit_synchronized(struct vi_info pi->link_cfg.link_ok = 0; pi->link_cfg.speed = 0; - pi->linkdnrc = -1; - t4_os_link_changed(sc, pi->port_id, 0, -1); + pi->link_cfg.link_down_rc = 255; + t4_os_link_changed(sc, pi->port_id, 0); return (0); } @@ -5274,8 +5300,14 @@ cxgbe_sysctls(struct port_info *pi) } SYSCTL_ADD_PROC(ctx, children, OID_AUTO, "pause_settings", - CTLTYPE_STRING | CTLFLAG_RW, pi, PAUSE_TX, sysctl_pause_settings, - "A", "PAUSE settings (bit 0 = rx_pause, bit 1 = tx_pause)"); + CTLTYPE_STRING | CTLFLAG_RW, pi, 0, sysctl_pause_settings, "A", + "PAUSE settings (bit 0 = rx_pause, bit 1 = tx_pause)"); + SYSCTL_ADD_PROC(ctx, children, OID_AUTO, "fec", + CTLTYPE_STRING | CTLFLAG_RW, pi, 0, sysctl_fec, "A", + "Forward Error Correction (bit 0 = RS, bit 1 = BASER_RS)"); + SYSCTL_ADD_PROC(ctx, children, OID_AUTO, "autoneg", + CTLTYPE_INT | CTLFLAG_RW, pi, 0, sysctl_autoneg, "I", + "autonegotiation (-1 = not supported)"); SYSCTL_ADD_INT(ctx, children, OID_AUTO, "max_speed", CTLFLAG_RD, NULL, port_top_speed(pi), "max speed (in Gbps)"); @@ -5735,12 +5767,9 @@ sysctl_pause_settings(SYSCTL_HANDLER_ARG if (rc) return (rc); if ((lc->requested_fc & (PAUSE_TX | PAUSE_RX)) != n) { - int link_ok = lc->link_ok; - lc->requested_fc &= ~(PAUSE_TX | PAUSE_RX); lc->requested_fc |= n; rc = -t4_link_l1cfg(sc, sc->mbox, pi->tx_chan, lc); - lc->link_ok = link_ok; /* restore */ } end_synchronized_op(sc, 0); } @@ -5749,6 +5778,97 @@ sysctl_pause_settings(SYSCTL_HANDLER_ARG } static int +sysctl_fec(SYSCTL_HANDLER_ARGS) +{ + struct port_info *pi = arg1; + struct adapter *sc = pi->adapter; + struct link_config *lc = &pi->link_cfg; + int rc; + + if (req->newptr == NULL) { + struct sbuf *sb; + static char *bits = "\20\1RS\2BASER_RS\3RESERVED"; + + rc = sysctl_wire_old_buffer(req, 0); + if (rc != 0) + return(rc); + + sb = sbuf_new_for_sysctl(NULL, NULL, 128, req); + if (sb == NULL) + return (ENOMEM); + + sbuf_printf(sb, "%b", lc->fec & M_FW_PORT_CAP_FEC, bits); + rc = sbuf_finish(sb); + sbuf_delete(sb); + } else { + char s[2]; + int n; + + s[0] = '0' + (lc->requested_fec & M_FW_PORT_CAP_FEC); + s[1] = 0; + + rc = sysctl_handle_string(oidp, s, sizeof(s), req); + if (rc != 0) + return(rc); + + if (s[1] != 0) + return (EINVAL); + if (s[0] < '0' || s[0] > '9') + return (EINVAL); /* not a number */ + n = s[0] - '0'; + if (n & ~M_FW_PORT_CAP_FEC) + return (EINVAL); /* some other bit is set too */ + + rc = begin_synchronized_op(sc, &pi->vi[0], SLEEP_OK | INTR_OK, + "t4fec"); + if (rc) + return (rc); + if ((lc->requested_fec & M_FW_PORT_CAP_FEC) != n) { + lc->requested_fec = n & + G_FW_PORT_CAP_FEC(lc->supported); + rc = -t4_link_l1cfg(sc, sc->mbox, pi->tx_chan, lc); + } + end_synchronized_op(sc, 0); + } + + return (rc); +} + +static int +sysctl_autoneg(SYSCTL_HANDLER_ARGS) +{ + struct port_info *pi = arg1; + struct adapter *sc = pi->adapter; + struct link_config *lc = &pi->link_cfg; + int rc, val, old; + + if (lc->supported & FW_PORT_CAP_ANEG) + val = lc->autoneg == AUTONEG_ENABLE ? 1 : 0; + else + val = -1; + rc = sysctl_handle_int(oidp, &val, 0, req); + if (rc != 0 || req->newptr == NULL) + return (rc); + if ((lc->supported & FW_PORT_CAP_ANEG) == 0) + return (ENOTSUP); + + val = val ? AUTONEG_ENABLE : AUTONEG_DISABLE; + if (lc->autoneg == val) + return (0); /* no change */ + + rc = begin_synchronized_op(sc, &pi->vi[0], SLEEP_OK | INTR_OK, + "t4aneg"); + if (rc) + return (rc); + old = lc->autoneg; + lc->autoneg = val; + rc = -t4_link_l1cfg(sc, sc->mbox, pi->tx_chan, lc); + if (rc != 0) + lc->autoneg = old; + return (rc); +} + +static int sysctl_handle_t4_reg64(SYSCTL_HANDLER_ARGS) { struct adapter *sc = arg1; @@ -6493,6 +6613,7 @@ sysctl_linkdnrc(SYSCTL_HANDLER_ARGS) { int rc = 0; struct port_info *pi = arg1; + struct link_config *lc = &pi->link_cfg; struct sbuf *sb; rc = sysctl_wire_old_buffer(req, 0); @@ -6502,10 +6623,10 @@ sysctl_linkdnrc(SYSCTL_HANDLER_ARGS) if (sb == NULL) return (ENOMEM); - if (pi->linkdnrc < 0) + if (lc->link_ok || lc->link_down_rc == 255) sbuf_printf(sb, "n/a"); else - sbuf_printf(sb, "%s", t4_link_down_rc_str(pi->linkdnrc)); + sbuf_printf(sb, "%s", t4_link_down_rc_str(lc->link_down_rc)); rc = sbuf_finish(sb); sbuf_delete(sb); @@ -8973,19 +9094,13 @@ t4_os_portmod_changed(const struct adapt } void -t4_os_link_changed(struct adapter *sc, int idx, int link_stat, int reason) +t4_os_link_changed(struct adapter *sc, int idx, int link_stat) { struct port_info *pi = sc->port[idx]; struct vi_info *vi; struct ifnet *ifp; int v; - if (link_stat) - pi->linkdnrc = -1; - else { - if (reason >= 0) - pi->linkdnrc = reason; - } for_each_vi(pi, v, vi) { ifp = vi->ifp; if (ifp == NULL) Modified: head/sys/dev/cxgbe/t4_vf.c ============================================================================== --- head/sys/dev/cxgbe/t4_vf.c Fri Dec 30 08:37:30 2016 (r310806) +++ head/sys/dev/cxgbe/t4_vf.c Fri Dec 30 08:59:49 2016 (r310807) @@ -668,8 +668,6 @@ t4vf_attach(device_t dev) n1g++; } - pi->linkdnrc = -1; - pi->dev = device_add_child(dev, sc->names->vf_ifnet_name, -1); if (pi->dev == NULL) { device_printf(dev,