From owner-svn-src-stable@FreeBSD.ORG Tue Feb 7 01:26:30 2012 Return-Path: Delivered-To: svn-src-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 084FE106566B; Tue, 7 Feb 2012 01:26:30 +0000 (UTC) (envelope-from np@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id E352A8FC0C; Tue, 7 Feb 2012 01:26:29 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.4/8.14.4) with ESMTP id q171QTCG062359; Tue, 7 Feb 2012 01:26:29 GMT (envelope-from np@svn.freebsd.org) Received: (from np@localhost) by svn.freebsd.org (8.14.4/8.14.4/Submit) id q171QT7C062353; Tue, 7 Feb 2012 01:26:29 GMT (envelope-from np@svn.freebsd.org) Message-Id: <201202070126.q171QT7C062353@svn.freebsd.org> From: Navdeep Parhar Date: Tue, 7 Feb 2012 01:26:29 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-8@freebsd.org X-SVN-Group: stable-8 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r231103 - in stable/8: share/man/man4 sys/dev/cxgbe sys/dev/cxgbe/common sys/dev/cxgbe/firmware sys/modules/cxgbe sys/modules/cxgbe/firmware tools/tools/cxgbetool X-BeenThere: svn-src-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SVN commit messages for all the -stable branches of the src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Feb 2012 01:26:30 -0000 Author: np Date: Tue Feb 7 01:26:29 2012 New Revision: 231103 URL: http://svn.freebsd.org/changeset/base/231103 Log: MFC r228491, r228561, r228594. r228491: Do not clobber the ingress queue's congestion setting. r228561: Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) r228594: Catch up with new driver ioctls in cxgbe. Added: stable/8/sys/dev/cxgbe/firmware/ - copied from r228561, head/sys/dev/cxgbe/firmware/ stable/8/sys/modules/cxgbe/firmware/ - copied from r228561, head/sys/modules/cxgbe/firmware/ Deleted: stable/8/sys/dev/cxgbe/common/t4fw_interface.h Modified: stable/8/share/man/man4/cxgbe.4 stable/8/sys/dev/cxgbe/adapter.h stable/8/sys/dev/cxgbe/common/common.h stable/8/sys/dev/cxgbe/common/t4_hw.c stable/8/sys/dev/cxgbe/common/t4_hw.h stable/8/sys/dev/cxgbe/offload.h stable/8/sys/dev/cxgbe/osdep.h stable/8/sys/dev/cxgbe/t4_ioctl.h stable/8/sys/dev/cxgbe/t4_l2t.c stable/8/sys/dev/cxgbe/t4_l2t.h stable/8/sys/dev/cxgbe/t4_main.c stable/8/sys/dev/cxgbe/t4_sge.c stable/8/sys/modules/cxgbe/Makefile stable/8/tools/tools/cxgbetool/cxgbetool.c Directory Properties: stable/8/share/man/man4/ (props changed) stable/8/sys/ (props changed) stable/8/tools/tools/ (props changed) Modified: stable/8/share/man/man4/cxgbe.4 ============================================================================== --- stable/8/share/man/man4/cxgbe.4 Mon Feb 6 21:50:11 2012 (r231102) +++ stable/8/share/man/man4/cxgbe.4 Tue Feb 7 01:26:29 2012 (r231103) @@ -99,18 +99,29 @@ Tunables can be set at the prompt before booting the kernel or stored in .Xr loader.conf 5 . .Bl -tag -width indent -.It Va hw.cxgbe.max_ntxq_10G_port -The maximum number of tx queues to use for a 10Gb port. -The default value is 8. -.It Va hw.cxgbe.max_nrxq_10G_port -The maximum number of rx queues to use for a 10Gb port. -The default value is 8. -.It Va hw.cxgbe.max_ntxq_1G_port -The maximum number of tx queues to use for a 1Gb port. -The default value is 2. -.It Va hw.cxgbe.max_nrxq_1G_port -The maximum number of rx queues to use for a 1Gb port. -The default value is 2. +.It Va hw.cxgbe.ntxq10g +The number of tx queues to use for a 10Gb port. The default is 16 or the number +of CPU cores in the system, whichever is less. +.It Va hw.cxgbe.nrxq10g +The number of rx queues to use for a 10Gb port. The default is 8 or the number +of CPU cores in the system, whichever is less. +.It Va hw.cxgbe.ntxq1g +The number of tx queues to use for a 1Gb port. The default is 4 or the number +of CPU cores in the system, whichever is less. +.It Va hw.cxgbe.nrxq1g +The number of rx queues to use for a 1Gb port. The default is 2 or the number +of CPU cores in the system, whichever is less. +.It Va hw.cxgbe.nofldtxq10g +The number of TOE tx queues to use for a 10Gb port. The default is 8 or the +number of CPU cores in the system, whichever is less. +.It Va hw.cxgbe.nofldrxq10g +The number of TOE rx queues to use for a 10Gb port. The default is 2 or the +number of CPU cores in the system, whichever is less. +.It Va hw.cxgbe.nofldtxq1g +The number of TOE tx queues to use for a 1Gb port. The default is 2 or the +number of CPU cores in the system, whichever is less. +.It Va hw.cxgbe.nofldrxq1g +The number of TOE rx queues to use for a 1Gb port. The default is 1. .It Va hw.cxgbe.holdoff_timer_idx_10G .It Va hw.cxgbe.holdoff_timer_idx_1G The timer index value to use to delay interrupts. @@ -119,6 +130,8 @@ by default (all values are in microsecon value from this list. The default value is 1 for both 10Gb and 1Gb ports, which means the timer value is 5us. +Different cxgbe interfaces can be assigned different values at any time via the +dev.cxgbe.X.holdoff_tmr_idx sysctl. .It Va hw.cxgbe.holdoff_pktc_idx_10G .It Va hw.cxgbe.holdoff_pktc_idx_1G The packet-count index value to use to delay interrupts. @@ -127,6 +140,11 @@ and the index selects a value from this The default value is 2 for both 10Gb and 1Gb ports, which means 16 packets (or the holdoff timer going off) before an interrupt is generated. +-1 disables packet counting. +Different cxgbe interfaces can be assigned different values via the +dev.cxgbe.X.holdoff_pktc_idx sysctl. +This sysctl works only when the interface has never been marked up (as done by +ifconfig up). .It Va hw.cxgbe.qsize_txq The size, in number of entries, of the descriptor ring used for a tx queue. @@ -134,10 +152,46 @@ A buf_ring of the same size is also allo software queuing. See .Xr ifnet 9 . The default value is 1024. +Different cxgbe interfaces can be assigned different values via the +dev.cxgbe.X.qsize_txq sysctl. +This sysctl works only when the interface has never been marked up (as done by +ifconfig up). .It Va hw.cxgbe.qsize_rxq The size, in number of entries, of the descriptor ring used for an rx queue. The default value is 1024. +Different cxgbe interfaces can be assigned different values via the +dev.cxgbe.X.qsize_rxq sysctl. +This sysctl works only when the interface has never been marked up (as done by +ifconfig up). +.It Va hw.cxgbe.interrupt_types +The interrupt types that the driver is allowed to use. +Bit 0 represents INTx (line interrupts), bit 1 MSI, bit 2 MSI-X. +The default is 7 (all allowed). +The driver will select the best possible type out of the allowed types by +itself. +.It Va hw.cxgbe.config_file +Select a pre-packaged device configuration file. +A configuration file contains a recipe for partitioning and configuring the +hardware resources on the card. +This tunable is for specialized applications only and should not be used in +normal operation. +The configuration profile currently in use is available in the dev.t4nex.X.cf +and dev.t4nex.X.cfcsum sysctls. +.It Va hw.cxgbe.linkcaps_allowed +.It Va hw.cxgbe.niccaps_allowed +.It Va hw.cxgbe.toecaps_allowed +.It Va hw.cxgbe.rdmacaps_allowed +.It Va hw.cxgbe.iscsicaps_allowed +.It Va hw.cxgbe.fcoecaps_allowed +Disallowing capabilities provides a hint to the driver and firmware to not +reserve hardware resources for that feature. +Each of these is a bit field with a bit for each sub-capability within the +capability. +This tunable is for specialized applications only and should not be used in +normal operation. +The capabilities for which hardware resources have been reserved are listed in +dev.t4nex.X.*caps sysctls. .El .Sh SUPPORT For general information and support, Modified: stable/8/sys/dev/cxgbe/adapter.h ============================================================================== --- stable/8/sys/dev/cxgbe/adapter.h Mon Feb 6 21:50:11 2012 (r231102) +++ stable/8/sys/dev/cxgbe/adapter.h Tue Feb 7 01:26:29 2012 (r231103) @@ -31,6 +31,7 @@ #ifndef __T4_ADAPTER_H__ #define __T4_ADAPTER_H__ +#include #include #include #include @@ -46,8 +47,9 @@ #include #include "offload.h" -#include "common/t4fw_interface.h" +#include "firmware/t4fw_interface.h" +#define T4_CFGNAME "t4fw_cfg" #define T4_FWNAME "t4fw" MALLOC_DECLARE(M_CXGBE); @@ -70,8 +72,9 @@ prefetch(void *x) #define CTLTYPE_U64 CTLTYPE_QUAD #endif -#if __FreeBSD_version >= 802507 -#define T4_DEVLOG 1 +#if (__FreeBSD_version >= 900030) || \ + ((__FreeBSD_version >= 802507) && (__FreeBSD_version < 900000)) +#define SBUF_DRAIN 1 #endif #ifdef __amd64__ @@ -120,25 +123,21 @@ enum { FW_IQ_QSIZE = 256, FW_IQ_ESIZE = 64, /* At least 64 mandated by the firmware spec */ - INTR_IQ_QSIZE = 64, - INTR_IQ_ESIZE = 64, /* Handles some CPLs too, do not reduce */ - - CTRL_EQ_QSIZE = 128, - CTRL_EQ_ESIZE = 64, - RX_IQ_QSIZE = 1024, RX_IQ_ESIZE = 64, /* At least 64 so CPL_RX_PKT will fit */ - RX_FL_ESIZE = 64, /* 8 64bit addresses */ + EQ_ESIZE = 64, /* All egress queues use this entry size */ + RX_FL_ESIZE = EQ_ESIZE, /* 8 64bit addresses */ #if MJUMPAGESIZE != MCLBYTES FL_BUF_SIZES = 4, /* cluster, jumbop, jumbo9k, jumbo16k */ #else FL_BUF_SIZES = 3, /* cluster, jumbo9k, jumbo16k */ #endif + CTRL_EQ_QSIZE = 128, + TX_EQ_QSIZE = 1024, - TX_EQ_ESIZE = 64, TX_SGL_SEGS = 36, TX_WR_FLITS = SGE_MAX_WR_LEN / 8 }; @@ -154,13 +153,16 @@ enum { /* adapter flags */ FULL_INIT_DONE = (1 << 0), FW_OK = (1 << 1), - INTR_SHARED = (1 << 2), /* one set of intrq's for all ports */ + INTR_DIRECT = (1 << 2), /* direct interrupts for everything */ + MASTER_PF = (1 << 3), + ADAP_SYSCTL_CTX = (1 << 4), CXGBE_BUSY = (1 << 9), /* port flags */ DOOMED = (1 << 0), - VI_ENABLED = (1 << 1), + PORT_INIT_DONE = (1 << 1), + PORT_SYSCTL_CTX = (1 << 2), }; #define IS_DOOMED(pi) (pi->flags & DOOMED) @@ -196,6 +198,12 @@ struct port_info { int first_txq; /* index of first tx queue */ int nrxq; /* # of rx queues */ int first_rxq; /* index of first rx queue */ +#ifndef TCP_OFFLOAD_DISABLE + int nofldtxq; /* # of offload tx queues */ + int first_ofld_txq; /* index of first offload tx queue */ + int nofldrxq; /* # of offload rx queues */ + int first_ofld_rxq; /* index of first offload rx queue */ +#endif int tmr_idx; int pktc_idx; int qsize_rxq; @@ -204,11 +212,8 @@ struct port_info { struct link_config link_cfg; struct port_stats stats; - struct taskqueue *tq; struct callout tick; - struct sysctl_ctx_list ctx; /* lives from ifconfig up to down */ - struct sysctl_oid *oid_rxq; - struct sysctl_oid *oid_txq; + struct sysctl_ctx_list ctx; /* from ifconfig up to driver detach */ uint8_t hw_addr[ETHER_ADDR_LEN]; /* factory MAC address, won't change */ }; @@ -232,17 +237,26 @@ struct tx_map { bus_dmamap_t map; }; +/* DMA maps used for tx */ +struct tx_maps { + struct tx_map *maps; + uint32_t map_total; /* # of DMA maps */ + uint32_t map_pidx; /* next map to be used */ + uint32_t map_cidx; /* reclaimed up to this index */ + uint32_t map_avail; /* # of available maps */ +}; + struct tx_sdesc { uint8_t desc_used; /* # of hardware descriptors used by the WR */ uint8_t credits; /* NIC txq: # of frames sent out in the WR */ }; -typedef void (iq_intr_handler_t)(void *); - enum { /* iq flags */ - IQ_ALLOCATED = (1 << 1), /* firmware resources allocated */ - IQ_STARTED = (1 << 2), /* started */ + IQ_ALLOCATED = (1 << 0), /* firmware resources allocated */ + IQ_HAS_FL = (1 << 1), /* iq associated with a freelist */ + IQ_INTR = (1 << 2), /* iq takes direct interrupt */ + IQ_LRO_ENABLED = (1 << 3), /* iq is an eth rxq with LRO enabled */ /* iq state */ IQS_DISABLED = 0, @@ -262,26 +276,35 @@ struct sge_iq { uint16_t abs_id; /* absolute SGE id for the iq */ int8_t intr_pktc_idx; /* packet count threshold index */ int8_t pad0; - iq_intr_handler_t *handler; __be64 *desc; /* KVA of descriptor ring */ - volatile uint32_t state; + volatile int state; struct adapter *adapter; const __be64 *cdesc; /* current descriptor */ uint8_t gen; /* generation bit */ uint8_t intr_params; /* interrupt holdoff parameters */ - uint8_t intr_next; /* holdoff for next interrupt */ + uint8_t intr_next; /* XXX: holdoff for next interrupt */ uint8_t esize; /* size (bytes) of each entry in the queue */ uint16_t qsize; /* size (# of entries) of the queue */ uint16_t cidx; /* consumer index */ - uint16_t cntxt_id; /* SGE context id for the iq */ + uint16_t cntxt_id; /* SGE context id for the iq */ + + STAILQ_ENTRY(sge_iq) link; }; enum { + EQ_CTRL = 1, + EQ_ETH = 2, +#ifndef TCP_OFFLOAD_DISABLE + EQ_OFLD = 3, +#endif + /* eq flags */ - EQ_ALLOCATED = (1 << 1), /* firmware resources allocated */ - EQ_STARTED = (1 << 2), /* started */ - EQ_CRFLUSHED = (1 << 3), /* expecting an update from SGE */ + EQ_TYPEMASK = 7, /* 3 lsbits hold the type */ + EQ_ALLOCATED = (1 << 3), /* firmware resources allocated */ + EQ_DOOMED = (1 << 4), /* about to be destroyed */ + EQ_CRFLUSHED = (1 << 5), /* expecting an update from SGE */ + EQ_STALLED = (1 << 6), /* out of hw descriptors or dmamaps */ }; /* @@ -291,10 +314,11 @@ enum { * consumes them) but it's special enough to have its own struct (see sge_fl). */ struct sge_eq { + unsigned int flags; /* MUST be first */ + unsigned int cntxt_id; /* SGE context id for the eq */ bus_dma_tag_t desc_tag; bus_dmamap_t desc_map; char lockname[16]; - unsigned int flags; struct mtx eq_lock; struct tx_desc *desc; /* KVA of descriptor ring */ @@ -307,9 +331,24 @@ struct sge_eq { uint16_t pidx; /* producer idx (desc idx) */ uint16_t pending; /* # of descriptors used since last doorbell */ uint16_t iqid; /* iq that gets egr_update for the eq */ - unsigned int cntxt_id; /* SGE context id for the eq */ + uint8_t tx_chan; /* tx channel used by the eq */ + struct task tx_task; + struct callout tx_callout; + + /* stats */ + + uint32_t egr_update; /* # of SGE_EGR_UPDATE notifications for eq */ + uint32_t unstalled; /* recovered from stall */ +}; + +enum { + FL_STARVING = (1 << 0), /* on the adapter's list of starving fl's */ + FL_DOOMED = (1 << 1), /* about to be destroyed */ }; +#define FL_RUNNING_LOW(fl) (fl->cap - fl->needed <= fl->lowat) +#define FL_NOT_RUNNING_LOW(fl) (fl->cap - fl->needed >= 2 * fl->lowat) + struct sge_fl { bus_dma_tag_t desc_tag; bus_dmamap_t desc_map; @@ -317,6 +356,7 @@ struct sge_fl { uint8_t tag_idx; struct mtx fl_lock; char lockname[16]; + int flags; __be64 *desc; /* KVA of descriptor ring, ptr to addresses */ bus_addr_t ba; /* bus address of descriptor ring */ @@ -327,8 +367,10 @@ struct sge_fl { uint32_t cidx; /* consumer idx (buffer idx, NOT hw desc idx) */ uint32_t pidx; /* producer idx (buffer idx, NOT hw desc idx) */ uint32_t needed; /* # of buffers needed to fill up fl. */ + uint32_t lowat; /* # of buffers <= this means fl needs help */ uint32_t pending; /* # of bufs allocated since last doorbell */ unsigned int dmamap_failed; + TAILQ_ENTRY(sge_fl) link; /* All starving freelists */ }; /* txq: SGE egress queue + what's needed for Ethernet NIC */ @@ -340,14 +382,8 @@ struct sge_txq { struct buf_ring *br; /* tx buffer ring */ struct tx_sdesc *sdesc; /* KVA of software descriptor ring */ struct mbuf *m; /* held up due to temporary resource shortage */ - struct task resume_tx; - /* DMA maps used for tx */ - struct tx_map *maps; - uint32_t map_total; /* # of DMA maps */ - uint32_t map_pidx; /* next map to be used */ - uint32_t map_cidx; /* reclaimed up to this index */ - uint32_t map_avail; /* # of available maps */ + struct tx_maps txmaps; /* stats for common events first */ @@ -364,20 +400,14 @@ struct sge_txq { uint32_t no_dmamap; /* no DMA map to load the mbuf */ uint32_t no_desc; /* out of hardware descriptors */ - uint32_t egr_update; /* # of SGE_EGR_UPDATE notifications for txq */ } __aligned(CACHE_LINE_SIZE); -enum { - RXQ_LRO_ENABLED = (1 << 0) -}; - /* rxq: SGE ingress queue + SGE free list + miscellaneous items */ struct sge_rxq { struct sge_iq iq; /* MUST be first */ - struct sge_fl fl; + struct sge_fl fl; /* MUST follow iq */ struct ifnet *ifp; /* the interface this rxq belongs to */ - unsigned int flags; #ifdef INET struct lro_ctrl lro; /* LRO state */ #endif @@ -391,12 +421,28 @@ struct sge_rxq { } __aligned(CACHE_LINE_SIZE); -/* ctrlq: SGE egress queue + stats for control queue */ -struct sge_ctrlq { +#ifndef TCP_OFFLOAD_DISABLE +/* ofld_rxq: SGE ingress queue + SGE free list + miscellaneous items */ +struct sge_ofld_rxq { + struct sge_iq iq; /* MUST be first */ + struct sge_fl fl; /* MUST follow iq */ +} __aligned(CACHE_LINE_SIZE); +#endif + +/* + * wrq: SGE egress queue that is given prebuilt work requests. Both the control + * and offload tx queues are of this type. + */ +struct sge_wrq { struct sge_eq eq; /* MUST be first */ + struct adapter *adapter; + struct mbuf *head; /* held up due to lack of descriptors */ + struct mbuf *tail; /* valid only if head is valid */ + /* stats for common events first */ + uint64_t tx_wrs; /* # of tx work requests */ /* stats for not-that-common events */ @@ -404,20 +450,28 @@ struct sge_ctrlq { } __aligned(CACHE_LINE_SIZE); struct sge { - uint16_t timer_val[SGE_NTIMERS]; - uint8_t counter_val[SGE_NCOUNTERS]; + int timer_val[SGE_NTIMERS]; + int counter_val[SGE_NCOUNTERS]; int fl_starve_threshold; - int nrxq; /* total rx queues (all ports and the rest) */ - int ntxq; /* total tx queues (all ports and the rest) */ - int niq; /* total ingress queues */ - int neq; /* total egress queues */ + int nrxq; /* total # of Ethernet rx queues */ + int ntxq; /* total # of Ethernet tx tx queues */ +#ifndef TCP_OFFLOAD_DISABLE + int nofldrxq; /* total # of TOE rx queues */ + int nofldtxq; /* total # of TOE tx queues */ +#endif + int niq; /* total # of ingress queues */ + int neq; /* total # of egress queues */ struct sge_iq fwq; /* Firmware event queue */ - struct sge_ctrlq *ctrlq;/* Control queues */ - struct sge_iq *intrq; /* Interrupt queues */ + struct sge_wrq mgmtq; /* Management queue (control queue) */ + struct sge_wrq *ctrlq; /* Control queues */ struct sge_txq *txq; /* NIC tx queues */ struct sge_rxq *rxq; /* NIC rx queues */ +#ifndef TCP_OFFLOAD_DISABLE + struct sge_wrq *ofld_txq; /* TOE tx queues */ + struct sge_ofld_rxq *ofld_rxq; /* TOE rx queues */ +#endif uint16_t iq_start; int eq_start; @@ -425,7 +479,12 @@ struct sge { struct sge_eq **eqmap; /* eq->cntxt_id to eq mapping */ }; +struct rss_header; +typedef int (*cpl_handler_t)(struct sge_iq *, const struct rss_header *, + struct mbuf *); + struct adapter { + SLIST_ENTRY(adapter) link; device_t dev; struct cdev *cdev; @@ -454,27 +513,47 @@ struct adapter { struct sge sge; + struct taskqueue *tq[NCHAN]; /* taskqueues that flush data out */ struct port_info *port[MAX_NPORTS]; uint8_t chan_map[NCHAN]; + uint32_t filter_mode; +#ifndef TCP_OFFLOAD_DISABLE + struct uld_softc tom; + struct tom_tunables tt; +#endif struct l2t_data *l2t; /* L2 table */ struct tid_info tids; - int registered_device_map; int open_device_map; +#ifndef TCP_OFFLOAD_DISABLE + int offload_map; +#endif int flags; char fw_version[32]; + unsigned int cfcsum; struct adapter_params params; struct t4_virt_res vres; - struct sysctl_ctx_list ctx; /* from first_port_up to last_port_down */ - struct sysctl_oid *oid_fwq; - struct sysctl_oid *oid_ctrlq; - struct sysctl_oid *oid_intrq; + uint16_t linkcaps; + uint16_t niccaps; + uint16_t toecaps; + uint16_t rdmacaps; + uint16_t iscsicaps; + uint16_t fcoecaps; + + struct sysctl_ctx_list ctx; /* from adapter_full_init to full_uninit */ struct mtx sc_lock; char lockname[16]; + + /* Starving free lists */ + struct mtx sfl_lock; /* same cache-line as sc_lock? but that's ok */ + TAILQ_HEAD(, sge_fl) sfl; + struct callout sfl_callout; + + cpl_handler_t cpl_handler[256] __aligned(CACHE_LINE_SIZE); }; #define ADAPTER_LOCK(sc) mtx_lock(&(sc)->sc_lock) @@ -516,11 +595,15 @@ struct adapter { #define for_each_rxq(pi, iter, rxq) \ rxq = &pi->adapter->sge.rxq[pi->first_rxq]; \ for (iter = 0; iter < pi->nrxq; ++iter, ++rxq) +#define for_each_ofld_txq(pi, iter, ofld_txq) \ + ofld_txq = &pi->adapter->sge.ofld_txq[pi->first_ofld_txq]; \ + for (iter = 0; iter < pi->nofldtxq; ++iter, ++ofld_txq) +#define for_each_ofld_rxq(pi, iter, ofld_rxq) \ + ofld_rxq = &pi->adapter->sge.ofld_rxq[pi->first_ofld_rxq]; \ + for (iter = 0; iter < pi->nofldrxq; ++iter, ++ofld_rxq) /* One for errors, one for firmware events */ #define T4_EXTRA_INTR 2 -#define NINTRQ(sc) ((sc)->intr_count > T4_EXTRA_INTR ? \ - (sc)->intr_count - T4_EXTRA_INTR : 1) static inline uint32_t t4_read_reg(struct adapter *sc, uint32_t reg) @@ -599,29 +682,52 @@ static inline bool is_10G_port(const str return ((pi->link_cfg.supported & FW_PORT_CAP_SPEED_10G) != 0); } +static inline int tx_resume_threshold(struct sge_eq *eq) +{ + return (eq->qsize / 4); +} + /* t4_main.c */ -void cxgbe_txq_start(void *, int); +void t4_tx_task(void *, int); +void t4_tx_callout(void *); int t4_os_find_pci_capability(struct adapter *, int); int t4_os_pci_save_state(struct adapter *); int t4_os_pci_restore_state(struct adapter *); void t4_os_portmod_changed(const struct adapter *, int); void t4_os_link_changed(struct adapter *, int, int); +void t4_iterate(void (*)(struct adapter *, void *), void *); +int t4_register_cpl_handler(struct adapter *, int, cpl_handler_t); /* t4_sge.c */ void t4_sge_modload(void); -void t4_sge_init(struct adapter *); +int t4_sge_init(struct adapter *); int t4_create_dma_tag(struct adapter *); int t4_destroy_dma_tag(struct adapter *); int t4_setup_adapter_queues(struct adapter *); int t4_teardown_adapter_queues(struct adapter *); -int t4_setup_eth_queues(struct port_info *); -int t4_teardown_eth_queues(struct port_info *); +int t4_setup_port_queues(struct port_info *); +int t4_teardown_port_queues(struct port_info *); +int t4_alloc_tx_maps(struct tx_maps *, bus_dma_tag_t, int, int); +void t4_free_tx_maps(struct tx_maps *, bus_dma_tag_t); void t4_intr_all(void *); void t4_intr(void *); void t4_intr_err(void *); void t4_intr_evt(void *); int t4_mgmt_tx(struct adapter *, struct mbuf *); +int t4_wrq_tx_locked(struct adapter *, struct sge_wrq *, struct mbuf *); int t4_eth_tx(struct ifnet *, struct sge_txq *, struct mbuf *); void t4_update_fl_bufsize(struct ifnet *); +int can_resume_tx(struct sge_eq *); + +static inline int t4_wrq_tx(struct adapter *sc, struct sge_wrq *wrq, struct mbuf *m) +{ + int rc; + + TXQ_LOCK(wrq); + rc = t4_wrq_tx_locked(sc, wrq, m); + TXQ_UNLOCK(wrq); + return (rc); +} + #endif Modified: stable/8/sys/dev/cxgbe/common/common.h ============================================================================== --- stable/8/sys/dev/cxgbe/common/common.h Mon Feb 6 21:50:11 2012 (r231102) +++ stable/8/sys/dev/cxgbe/common/common.h Tue Feb 7 01:26:29 2012 (r231103) @@ -42,6 +42,15 @@ enum { enum { MEM_EDC0, MEM_EDC1, MEM_MC }; +enum { + MEMWIN0_APERTURE = 2048, + MEMWIN0_BASE = 0x1b800, + MEMWIN1_APERTURE = 32768, + MEMWIN1_BASE = 0x28000, + MEMWIN2_APERTURE = 65536, + MEMWIN2_BASE = 0x30000, +}; + enum dev_master { MASTER_CANT, MASTER_MAY, MASTER_MUST }; enum dev_state { DEV_STATE_UNINIT, DEV_STATE_INIT, DEV_STATE_ERR }; @@ -53,8 +62,8 @@ enum { }; #define FW_VERSION_MAJOR 1 -#define FW_VERSION_MINOR 3 -#define FW_VERSION_MICRO 10 +#define FW_VERSION_MINOR 4 +#define FW_VERSION_MICRO 16 struct port_stats { u64 tx_octets; /* total # of octets in good frames */ @@ -190,7 +199,6 @@ struct tp_proxy_stats { struct tp_cpl_stats { u32 req[4]; u32 rsp[4]; - u32 tx_err[4]; }; struct tp_rdma_stats { @@ -214,9 +222,9 @@ struct vpd_params { }; struct pci_params { - unsigned int vpd_cap_addr; - unsigned char speed; - unsigned char width; + unsigned int vpd_cap_addr; + unsigned short speed; + unsigned short width; }; /* @@ -239,20 +247,20 @@ struct adapter_params { unsigned int fw_vers; unsigned int tp_vers; - u8 api_vers[7]; unsigned short mtus[NMTUS]; unsigned short a_wnd[NCCTRL_WIN]; unsigned short b_wnd[NCCTRL_WIN]; - unsigned int mc_size; /* MC memory size */ - unsigned int nfilters; /* size of filter region */ + unsigned int mc_size; /* MC memory size */ + unsigned int nfilters; /* size of filter region */ unsigned int cim_la_size; - unsigned int nports; /* # of ethernet ports */ + /* Used as int in sysctls, do not reduce size */ + unsigned int nports; /* # of ethernet ports */ unsigned int portvec; - unsigned int rev; /* chip revision */ + unsigned int rev; /* chip revision */ unsigned int offload; unsigned int ofldq_wr_cred; @@ -366,6 +374,9 @@ int t4_seeprom_wp(struct adapter *adapte int t4_read_flash(struct adapter *adapter, unsigned int addr, unsigned int nwords, u32 *data, int byte_oriented); int t4_load_fw(struct adapter *adapter, const u8 *fw_data, unsigned int size); +int t4_load_boot(struct adapter *adap, const u8 *boot_data, + unsigned int boot_addr, unsigned int size); +unsigned int t4_flash_cfg_addr(struct adapter *adapter); int t4_load_cfg(struct adapter *adapter, const u8 *cfg_data, unsigned int size); int t4_get_fw_version(struct adapter *adapter, u32 *vers); int t4_get_tp_version(struct adapter *adapter, u32 *vers); @@ -460,8 +471,8 @@ int t4_wol_pat_enable(struct adapter *ad int t4_fw_hello(struct adapter *adap, unsigned int mbox, unsigned int evt_mbox, enum dev_master master, enum dev_state *state); int t4_fw_bye(struct adapter *adap, unsigned int mbox); -int t4_early_init(struct adapter *adap, unsigned int mbox); int t4_fw_reset(struct adapter *adap, unsigned int mbox, int reset); +int t4_fw_initialize(struct adapter *adap, unsigned int mbox); int t4_query_params(struct adapter *adap, unsigned int mbox, unsigned int pf, unsigned int vf, unsigned int nparams, const u32 *params, u32 *val); Modified: stable/8/sys/dev/cxgbe/common/t4_hw.c ============================================================================== --- stable/8/sys/dev/cxgbe/common/t4_hw.c Mon Feb 6 21:50:11 2012 (r231102) +++ stable/8/sys/dev/cxgbe/common/t4_hw.c Tue Feb 7 01:26:29 2012 (r231103) @@ -30,10 +30,10 @@ __FBSDID("$FreeBSD$"); #include "common.h" #include "t4_regs.h" #include "t4_regs_values.h" -#include "t4fw_interface.h" +#include "firmware/t4fw_interface.h" #undef msleep -#define msleep(x) DELAY((x) * 1000) +#define msleep(x) pause("t4hw", (x) * hz / 1000) /** * t4_wait_op_done_val - wait until an operation is completed @@ -187,7 +187,7 @@ int t4_wr_mbox_meat(struct adapter *adap * off to larger delays to a maximum retry delay. */ static const int delay[] = { - 1, 1, 3, 5, 10, 10, 20, 50, 100, 200 + 1, 1, 3, 5, 10, 10, 20, 50, 100 }; u32 v; @@ -625,17 +625,6 @@ enum { SF_RD_DATA_FAST = 0xb, /* read flash */ SF_RD_ID = 0x9f, /* read ID */ SF_ERASE_SECTOR = 0xd8, /* erase sector */ - - FW_START_SEC = 8, /* first flash sector for FW */ - FW_END_SEC = 15, /* last flash sector for FW */ - FW_IMG_START = FW_START_SEC * SF_SEC_SIZE, - FW_MAX_SIZE = (FW_END_SEC - FW_START_SEC + 1) * SF_SEC_SIZE, - - FLASH_CFG_MAX_SIZE = 0x10000 , /* max size of the flash config file */ - FLASH_CFG_OFFSET = 0x1f0000, - FLASH_CFG_START_SEC = FLASH_CFG_OFFSET / SF_SEC_SIZE, - FPGA_FLASH_CFG_OFFSET = 0xf0000 , /* if FPGA mode, then cfg file is at 1MB - 64KB */ - FPGA_FLASH_CFG_START_SEC = FPGA_FLASH_CFG_OFFSET / SF_SEC_SIZE, }; /** @@ -763,12 +752,15 @@ int t4_read_flash(struct adapter *adapte * @addr: the start address to write * @n: length of data to write in bytes * @data: the data to write + * @byte_oriented: whether to store data as bytes or as words * * Writes up to a page of data (256 bytes) to the serial flash starting * at the given address. All the data must be written to the same page. + * If @byte_oriented is set the write data is stored as byte stream + * (i.e. matches what on disk), otherwise in big-endian. */ static int t4_write_flash(struct adapter *adapter, unsigned int addr, - unsigned int n, const u8 *data) + unsigned int n, const u8 *data, int byte_oriented) { int ret; u32 buf[SF_PAGE_SIZE / 4]; @@ -788,6 +780,9 @@ static int t4_write_flash(struct adapter for (val = 0, i = 0; i < c; ++i) val = (val << 8) + *data++; + if (!byte_oriented) + val = htonl(val); + ret = sf1_write(adapter, c, c != left, 1, val); if (ret) goto unlock; @@ -799,7 +794,8 @@ static int t4_write_flash(struct adapter t4_write_reg(adapter, A_SF_OP, 0); /* unlock SF */ /* Read the page to verify the write succeeded */ - ret = t4_read_flash(adapter, addr & ~0xff, ARRAY_SIZE(buf), buf, 1); + ret = t4_read_flash(adapter, addr & ~0xff, ARRAY_SIZE(buf), buf, + byte_oriented); if (ret) return ret; @@ -825,7 +821,7 @@ unlock: int t4_get_fw_version(struct adapter *adapter, u32 *vers) { return t4_read_flash(adapter, - FW_IMG_START + offsetof(struct fw_hdr, fw_ver), 1, + FLASH_FW_START + offsetof(struct fw_hdr, fw_ver), 1, vers, 0); } @@ -838,7 +834,7 @@ int t4_get_fw_version(struct adapter *ad */ int t4_get_tp_version(struct adapter *adapter, u32 *vers) { - return t4_read_flash(adapter, FW_IMG_START + offsetof(struct fw_hdr, + return t4_read_flash(adapter, FLASH_FW_START + offsetof(struct fw_hdr, tp_microcode_ver), 1, vers, 0); } @@ -854,24 +850,17 @@ int t4_get_tp_version(struct adapter *ad */ int t4_check_fw_version(struct adapter *adapter) { - u32 api_vers[2]; int ret, major, minor, micro; ret = t4_get_fw_version(adapter, &adapter->params.fw_vers); if (!ret) ret = t4_get_tp_version(adapter, &adapter->params.tp_vers); - if (!ret) - ret = t4_read_flash(adapter, - FW_IMG_START + offsetof(struct fw_hdr, intfver_nic), - 2, api_vers, 1); if (ret) return ret; major = G_FW_HDR_FW_VER_MAJOR(adapter->params.fw_vers); minor = G_FW_HDR_FW_VER_MINOR(adapter->params.fw_vers); micro = G_FW_HDR_FW_VER_MICRO(adapter->params.fw_vers); - memcpy(adapter->params.api_vers, api_vers, - sizeof(adapter->params.api_vers)); if (major != FW_VERSION_MAJOR) { /* major mismatch - fail */ CH_ERR(adapter, "card FW has major version %u, driver wants " @@ -914,6 +903,21 @@ static int t4_flash_erase_sectors(struct } /** + * t4_flash_cfg_addr - return the address of the flash configuration file + * @adapter: the adapter + * + * Return the address within the flash where the Firmware Configuration + * File is stored. + */ +unsigned int t4_flash_cfg_addr(struct adapter *adapter) +{ + if (adapter->params.sf_size == 0x100000) + return FLASH_FPGA_CFG_START; + else + return FLASH_CFG_START; +} + +/** * t4_load_cfg - download config file * @adap: the adapter * @cfg_data: the cfg text file to write @@ -928,17 +932,8 @@ int t4_load_cfg(struct adapter *adap, co unsigned int flash_cfg_start_sec; unsigned int sf_sec_size = adap->params.sf_size / adap->params.sf_nsec; - if (adap->params.sf_size == 0x100000) { - addr = FPGA_FLASH_CFG_OFFSET; - flash_cfg_start_sec = FPGA_FLASH_CFG_START_SEC; - } else { - addr = FLASH_CFG_OFFSET; - flash_cfg_start_sec = FLASH_CFG_START_SEC; - } - if (!size) { - CH_ERR(adap, "cfg file has no data\n"); - return -EINVAL; - } + addr = t4_flash_cfg_addr(adap); + flash_cfg_start_sec = addr / SF_SEC_SIZE; if (size > FLASH_CFG_MAX_SIZE) { CH_ERR(adap, "cfg file too large, max is %u bytes\n", @@ -950,7 +945,11 @@ int t4_load_cfg(struct adapter *adap, co sf_sec_size); ret = t4_flash_erase_sectors(adap, flash_cfg_start_sec, flash_cfg_start_sec + i - 1); - if (ret) + /* + * If size == 0 then we're simply erasing the FLASH sectors associated + * with the on-adapter Firmware Configuration File. + */ + if (ret || size == 0) goto out; /* this will write to the flash up to SF_PAGE_SIZE at a time */ @@ -959,7 +958,7 @@ int t4_load_cfg(struct adapter *adap, co n = size - i; else n = SF_PAGE_SIZE; - ret = t4_write_flash(adap, addr, n, cfg_data); + ret = t4_write_flash(adap, addr, n, cfg_data, 1); if (ret) goto out; @@ -969,7 +968,8 @@ int t4_load_cfg(struct adapter *adap, co out: if (ret) - CH_ERR(adap, "config file download failed %d\n", ret); + CH_ERR(adap, "config file %s failed %d\n", + (size == 0 ? "clear" : "download"), ret); return ret; } @@ -1004,9 +1004,9 @@ int t4_load_fw(struct adapter *adap, con CH_ERR(adap, "FW image size differs from size in FW header\n"); return -EINVAL; } - if (size > FW_MAX_SIZE) { + if (size > FLASH_FW_MAX_SIZE) { CH_ERR(adap, "FW image too large, max is %u bytes\n", - FW_MAX_SIZE); + FLASH_FW_MAX_SIZE); return -EFBIG; } @@ -1020,7 +1020,8 @@ int t4_load_fw(struct adapter *adap, con } i = DIV_ROUND_UP(size, sf_sec_size); /* # of sectors spanned */ - ret = t4_flash_erase_sectors(adap, FW_START_SEC, FW_START_SEC + i - 1); + ret = t4_flash_erase_sectors(adap, FLASH_FW_START_SEC, + FLASH_FW_START_SEC + i - 1); if (ret) goto out; @@ -1031,28 +1032,110 @@ int t4_load_fw(struct adapter *adap, con */ memcpy(first_page, fw_data, SF_PAGE_SIZE); ((struct fw_hdr *)first_page)->fw_ver = htonl(0xffffffff); - ret = t4_write_flash(adap, FW_IMG_START, SF_PAGE_SIZE, first_page); + ret = t4_write_flash(adap, FLASH_FW_START, SF_PAGE_SIZE, first_page, 1); if (ret) goto out; - addr = FW_IMG_START; + addr = FLASH_FW_START; for (size -= SF_PAGE_SIZE; size; size -= SF_PAGE_SIZE) { addr += SF_PAGE_SIZE; fw_data += SF_PAGE_SIZE; - ret = t4_write_flash(adap, addr, SF_PAGE_SIZE, fw_data); + ret = t4_write_flash(adap, addr, SF_PAGE_SIZE, fw_data, 1); if (ret) goto out; } ret = t4_write_flash(adap, - FW_IMG_START + offsetof(struct fw_hdr, fw_ver), - sizeof(hdr->fw_ver), (const u8 *)&hdr->fw_ver); + FLASH_FW_START + offsetof(struct fw_hdr, fw_ver), + sizeof(hdr->fw_ver), (const u8 *)&hdr->fw_ver, 1); out: if (ret) CH_ERR(adap, "firmware download failed, error %d\n", ret); return ret; } +/* BIOS boot header */ +typedef struct boot_header_s { + u8 signature[2]; /* signature */ + u8 length; /* image length (include header) */ + u8 offset[4]; /* initialization vector */ + u8 reserved[19]; /* reserved */ + u8 exheader[2]; /* offset to expansion header */ +} boot_header_t; + +enum { + BOOT_FLASH_BOOT_ADDR = 0x0,/* start address of boot image in flash */ + BOOT_SIGNATURE = 0xaa55, /* signature of BIOS boot ROM */ + BOOT_SIZE_INC = 512, /* image size measured in 512B chunks */ + BOOT_MIN_SIZE = sizeof(boot_header_t), /* at least basic header */ + BOOT_MAX_SIZE = 1024*BOOT_SIZE_INC /* 1 byte * length increment */ +}; + +/* + * t4_load_boot - download boot flash + * @adapter: the adapter + * @boot_data: the boot image to write + * @size: image size + * + * Write the supplied boot image to the card's serial flash. + * The boot image has the following sections: a 28-byte header and the + * boot image. + */ +int t4_load_boot(struct adapter *adap, const u8 *boot_data, + unsigned int boot_addr, unsigned int size) +{ + int ret, addr; + unsigned int i; + unsigned int boot_sector = boot_addr * 1024; + unsigned int sf_sec_size = adap->params.sf_size / adap->params.sf_nsec; + + /* + * Perform some primitive sanity testing to avoid accidentally + * writing garbage over the boot sectors. We ought to check for + * more but it's not worth it for now ... + */ + if (size < BOOT_MIN_SIZE || size > BOOT_MAX_SIZE) { + CH_ERR(adap, "boot image too small/large\n"); + return -EFBIG; + } + + /* + * Make sure the boot image does not encroach on the firmware region + */ + if ((boot_sector + size) >> 16 > FLASH_FW_START_SEC) { + CH_ERR(adap, "boot image encroaching on firmware region\n"); + return -EFBIG; + } + + i = DIV_ROUND_UP(size, sf_sec_size); /* # of sectors spanned */ + ret = t4_flash_erase_sectors(adap, boot_sector >> 16, + (boot_sector >> 16) + i - 1); *** DIFF OUTPUT TRUNCATED AT 1000 LINES ***