Skip site navigation (1)Skip section navigation (2)
Date:      Tue, 7 Feb 2012 01:26:29 +0000 (UTC)
From:      Navdeep Parhar <np@FreeBSD.org>
To:        src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-8@freebsd.org
Subject:   svn commit: r231103 - in stable/8: share/man/man4 sys/dev/cxgbe sys/dev/cxgbe/common sys/dev/cxgbe/firmware sys/modules/cxgbe sys/modules/cxgbe/firmware tools/tools/cxgbetool
Message-ID:  <201202070126.q171QT7C062353@svn.freebsd.org>

next in thread | raw e-mail | index | archive | help
Author: np
Date: Tue Feb  7 01:26:29 2012
New Revision: 231103
URL: http://svn.freebsd.org/changeset/base/231103

Log:
  MFC r228491, r228561, r228594.
  
  r228491:
  Do not clobber the ingress queue's congestion setting.
  
  r228561:
  Many updates to cxgbe(4)
  
  - Device configuration via plain text config file.  Also able to operate
    when not attached to the chip as the master driver.
  
  - Generic "work request" queue that serves as the base for both ctrl and
    ofld tx queues.
  
  - Generic interrupt handler routine that can process any event on any
    kind of ingress queue (via a dispatch table).
  
  - A couple of new driver ioctls.  cxgbetool can now install a firmware
    to the card ("loadfw" command) and can read the card's memory
    ("memdump" and "tcb" commands).
  
  - Lots of assorted information within dev.t4nex.X.misc.*  This is
    primarily for debugging and won't show up in sysctl -a.
  
  - Code to manage the L2 tables on the chip.
  
  - Updates to cxgbe(4) man page to go with the tunables that have changed.
  
  - Updates to the shared code in common/
  
  - Updates to the driver-firmware interface (now at fw 1.4.16.0)
  
  r228594:
  Catch up with new driver ioctls in cxgbe.

Added:
  stable/8/sys/dev/cxgbe/firmware/
     - copied from r228561, head/sys/dev/cxgbe/firmware/
  stable/8/sys/modules/cxgbe/firmware/
     - copied from r228561, head/sys/modules/cxgbe/firmware/
Deleted:
  stable/8/sys/dev/cxgbe/common/t4fw_interface.h
Modified:
  stable/8/share/man/man4/cxgbe.4
  stable/8/sys/dev/cxgbe/adapter.h
  stable/8/sys/dev/cxgbe/common/common.h
  stable/8/sys/dev/cxgbe/common/t4_hw.c
  stable/8/sys/dev/cxgbe/common/t4_hw.h
  stable/8/sys/dev/cxgbe/offload.h
  stable/8/sys/dev/cxgbe/osdep.h
  stable/8/sys/dev/cxgbe/t4_ioctl.h
  stable/8/sys/dev/cxgbe/t4_l2t.c
  stable/8/sys/dev/cxgbe/t4_l2t.h
  stable/8/sys/dev/cxgbe/t4_main.c
  stable/8/sys/dev/cxgbe/t4_sge.c
  stable/8/sys/modules/cxgbe/Makefile
  stable/8/tools/tools/cxgbetool/cxgbetool.c
Directory Properties:
  stable/8/share/man/man4/   (props changed)
  stable/8/sys/   (props changed)
  stable/8/tools/tools/   (props changed)

Modified: stable/8/share/man/man4/cxgbe.4
==============================================================================
--- stable/8/share/man/man4/cxgbe.4	Mon Feb  6 21:50:11 2012	(r231102)
+++ stable/8/share/man/man4/cxgbe.4	Tue Feb  7 01:26:29 2012	(r231103)
@@ -99,18 +99,29 @@ Tunables can be set at the
 prompt before booting the kernel or stored in
 .Xr loader.conf 5 .
 .Bl -tag -width indent
-.It Va hw.cxgbe.max_ntxq_10G_port
-The maximum number of tx queues to use for a 10Gb port.
-The default value is 8.
-.It Va hw.cxgbe.max_nrxq_10G_port
-The maximum number of rx queues to use for a 10Gb port.
-The default value is 8.
-.It Va hw.cxgbe.max_ntxq_1G_port
-The maximum number of tx queues to use for a 1Gb port.
-The default value is 2.
-.It Va hw.cxgbe.max_nrxq_1G_port
-The maximum number of rx queues to use for a 1Gb port.
-The default value is 2.
+.It Va hw.cxgbe.ntxq10g
+The number of tx queues to use for a 10Gb port.  The default is 16 or the number
+of CPU cores in the system, whichever is less.
+.It Va hw.cxgbe.nrxq10g
+The number of rx queues to use for a 10Gb port.  The default is 8 or the number
+of CPU cores in the system, whichever is less.
+.It Va hw.cxgbe.ntxq1g
+The number of tx queues to use for a 1Gb port.  The default is 4 or the number
+of CPU cores in the system, whichever is less.
+.It Va hw.cxgbe.nrxq1g
+The number of rx queues to use for a 1Gb port.  The default is 2 or the number
+of CPU cores in the system, whichever is less.
+.It Va hw.cxgbe.nofldtxq10g
+The number of TOE tx queues to use for a 10Gb port.  The default is 8 or the
+number of CPU cores in the system, whichever is less.
+.It Va hw.cxgbe.nofldrxq10g
+The number of TOE rx queues to use for a 10Gb port.  The default is 2 or the
+number of CPU cores in the system, whichever is less.
+.It Va hw.cxgbe.nofldtxq1g
+The number of TOE tx queues to use for a 1Gb port.  The default is 2 or the
+number of CPU cores in the system, whichever is less.
+.It Va hw.cxgbe.nofldrxq1g
+The number of TOE rx queues to use for a 1Gb port.  The default is 1.
 .It Va hw.cxgbe.holdoff_timer_idx_10G
 .It Va hw.cxgbe.holdoff_timer_idx_1G
 The timer index value to use to delay interrupts.
@@ -119,6 +130,8 @@ by default (all values are in microsecon
 value from this list.
 The default value is 1 for both 10Gb and 1Gb ports, which means the
 timer value is 5us.
+Different cxgbe interfaces can be assigned different values at any time via the
+dev.cxgbe.X.holdoff_tmr_idx sysctl.
 .It Va hw.cxgbe.holdoff_pktc_idx_10G
 .It Va hw.cxgbe.holdoff_pktc_idx_1G
 The packet-count index value to use to delay interrupts.
@@ -127,6 +140,11 @@ and the index selects a value from this 
 The default value is 2 for both 10Gb and 1Gb ports, which means 16
 packets (or the holdoff timer going off) before an interrupt is
 generated.
+-1 disables packet counting.
+Different cxgbe interfaces can be assigned different values via the
+dev.cxgbe.X.holdoff_pktc_idx sysctl.
+This sysctl works only when the interface has never been marked up (as done by
+ifconfig up).
 .It Va hw.cxgbe.qsize_txq
 The size, in number of entries, of the descriptor ring used for a tx
 queue.
@@ -134,10 +152,46 @@ A buf_ring of the same size is also allo
 software queuing.  See
 .Xr ifnet 9 .
 The default value is 1024.
+Different cxgbe interfaces can be assigned different values via the
+dev.cxgbe.X.qsize_txq sysctl.
+This sysctl works only when the interface has never been marked up (as done by
+ifconfig up).
 .It Va hw.cxgbe.qsize_rxq
 The size, in number of entries, of the descriptor ring used for an
 rx queue.
 The default value is 1024.
+Different cxgbe interfaces can be assigned different values via the
+dev.cxgbe.X.qsize_rxq sysctl.
+This sysctl works only when the interface has never been marked up (as done by
+ifconfig up).
+.It Va hw.cxgbe.interrupt_types
+The interrupt types that the driver is allowed to use.
+Bit 0 represents INTx (line interrupts), bit 1 MSI, bit 2 MSI-X.
+The default is 7 (all allowed).
+The driver will select the best possible type out of the allowed types by
+itself.
+.It Va hw.cxgbe.config_file
+Select a pre-packaged device configuration file.
+A configuration file contains a recipe for partitioning and configuring the
+hardware resources on the card.
+This tunable is for specialized applications only and should not be used in
+normal operation.
+The configuration profile currently in use is available in the dev.t4nex.X.cf
+and dev.t4nex.X.cfcsum sysctls.
+.It Va hw.cxgbe.linkcaps_allowed
+.It Va hw.cxgbe.niccaps_allowed
+.It Va hw.cxgbe.toecaps_allowed
+.It Va hw.cxgbe.rdmacaps_allowed
+.It Va hw.cxgbe.iscsicaps_allowed
+.It Va hw.cxgbe.fcoecaps_allowed
+Disallowing capabilities provides a hint to the driver and firmware to not
+reserve hardware resources for that feature.
+Each of these is a bit field with a bit for each sub-capability within the
+capability.
+This tunable is for specialized applications only and should not be used in
+normal operation.
+The capabilities for which hardware resources have been reserved are listed in
+dev.t4nex.X.*caps sysctls.
 .El
 .Sh SUPPORT
 For general information and support,

Modified: stable/8/sys/dev/cxgbe/adapter.h
==============================================================================
--- stable/8/sys/dev/cxgbe/adapter.h	Mon Feb  6 21:50:11 2012	(r231102)
+++ stable/8/sys/dev/cxgbe/adapter.h	Tue Feb  7 01:26:29 2012	(r231103)
@@ -31,6 +31,7 @@
 #ifndef __T4_ADAPTER_H__
 #define __T4_ADAPTER_H__
 
+#include <sys/kernel.h>
 #include <sys/bus.h>
 #include <sys/rman.h>
 #include <sys/types.h>
@@ -46,8 +47,9 @@
 #include <netinet/tcp_lro.h>
 
 #include "offload.h"
-#include "common/t4fw_interface.h"
+#include "firmware/t4fw_interface.h"
 
+#define T4_CFGNAME "t4fw_cfg"
 #define T4_FWNAME "t4fw"
 
 MALLOC_DECLARE(M_CXGBE);
@@ -70,8 +72,9 @@ prefetch(void *x)
 #define CTLTYPE_U64 CTLTYPE_QUAD
 #endif
 
-#if __FreeBSD_version >= 802507
-#define T4_DEVLOG 1
+#if (__FreeBSD_version >= 900030) || \
+    ((__FreeBSD_version >= 802507) && (__FreeBSD_version < 900000))
+#define SBUF_DRAIN 1
 #endif
 
 #ifdef __amd64__
@@ -120,25 +123,21 @@ enum {
 	FW_IQ_QSIZE = 256,
 	FW_IQ_ESIZE = 64,	/* At least 64 mandated by the firmware spec */
 
-	INTR_IQ_QSIZE = 64,
-	INTR_IQ_ESIZE = 64,	/* Handles some CPLs too, do not reduce */
-
-	CTRL_EQ_QSIZE = 128,
-	CTRL_EQ_ESIZE = 64,
-
 	RX_IQ_QSIZE = 1024,
 	RX_IQ_ESIZE = 64,	/* At least 64 so CPL_RX_PKT will fit */
 
-	RX_FL_ESIZE = 64,	/* 8 64bit addresses */
+	EQ_ESIZE = 64,		/* All egress queues use this entry size */
 
+	RX_FL_ESIZE = EQ_ESIZE,	/* 8 64bit addresses */
 #if MJUMPAGESIZE != MCLBYTES
 	FL_BUF_SIZES = 4,	/* cluster, jumbop, jumbo9k, jumbo16k */
 #else
 	FL_BUF_SIZES = 3,	/* cluster, jumbo9k, jumbo16k */
 #endif
 
+	CTRL_EQ_QSIZE = 128,
+
 	TX_EQ_QSIZE = 1024,
-	TX_EQ_ESIZE = 64,
 	TX_SGL_SEGS = 36,
 	TX_WR_FLITS = SGE_MAX_WR_LEN / 8
 };
@@ -154,13 +153,16 @@ enum {
 	/* adapter flags */
 	FULL_INIT_DONE	= (1 << 0),
 	FW_OK		= (1 << 1),
-	INTR_SHARED	= (1 << 2),	/* one set of intrq's for all ports */
+	INTR_DIRECT	= (1 << 2),	/* direct interrupts for everything */
+	MASTER_PF	= (1 << 3),
+	ADAP_SYSCTL_CTX	= (1 << 4),
 
 	CXGBE_BUSY	= (1 << 9),
 
 	/* port flags */
 	DOOMED		= (1 << 0),
-	VI_ENABLED	= (1 << 1),
+	PORT_INIT_DONE	= (1 << 1),
+	PORT_SYSCTL_CTX	= (1 << 2),
 };
 
 #define IS_DOOMED(pi)	(pi->flags & DOOMED)
@@ -196,6 +198,12 @@ struct port_info {
 	int first_txq;	/* index of first tx queue */
 	int nrxq;	/* # of rx queues */
 	int first_rxq;	/* index of first rx queue */
+#ifndef TCP_OFFLOAD_DISABLE
+	int nofldtxq;		/* # of offload tx queues */
+	int first_ofld_txq;	/* index of first offload tx queue */
+	int nofldrxq;		/* # of offload rx queues */
+	int first_ofld_rxq;	/* index of first offload rx queue */
+#endif
 	int tmr_idx;
 	int pktc_idx;
 	int qsize_rxq;
@@ -204,11 +212,8 @@ struct port_info {
 	struct link_config link_cfg;
 	struct port_stats stats;
 
-	struct taskqueue *tq;
 	struct callout tick;
-	struct sysctl_ctx_list ctx;	/* lives from ifconfig up to down */
-	struct sysctl_oid *oid_rxq;
-	struct sysctl_oid *oid_txq;
+	struct sysctl_ctx_list ctx;	/* from ifconfig up to driver detach */
 
 	uint8_t hw_addr[ETHER_ADDR_LEN]; /* factory MAC address, won't change */
 };
@@ -232,17 +237,26 @@ struct tx_map {
 	bus_dmamap_t map;
 };
 
+/* DMA maps used for tx */
+struct tx_maps {
+	struct tx_map *maps;
+	uint32_t map_total;	/* # of DMA maps */
+	uint32_t map_pidx;	/* next map to be used */
+	uint32_t map_cidx;	/* reclaimed up to this index */
+	uint32_t map_avail;	/* # of available maps */
+};
+
 struct tx_sdesc {
 	uint8_t desc_used;	/* # of hardware descriptors used by the WR */
 	uint8_t credits;	/* NIC txq: # of frames sent out in the WR */
 };
 
-typedef void (iq_intr_handler_t)(void *);
-
 enum {
 	/* iq flags */
-	IQ_ALLOCATED	= (1 << 1),	/* firmware resources allocated */
-	IQ_STARTED	= (1 << 2),	/* started */
+	IQ_ALLOCATED	= (1 << 0),	/* firmware resources allocated */
+	IQ_HAS_FL	= (1 << 1),	/* iq associated with a freelist */
+	IQ_INTR		= (1 << 2),	/* iq takes direct interrupt */
+	IQ_LRO_ENABLED	= (1 << 3),	/* iq is an eth rxq with LRO enabled */
 
 	/* iq state */
 	IQS_DISABLED	= 0,
@@ -262,26 +276,35 @@ struct sge_iq {
 	uint16_t abs_id;	/* absolute SGE id for the iq */
 	int8_t   intr_pktc_idx;	/* packet count threshold index */
 	int8_t   pad0;
-	iq_intr_handler_t *handler;
 	__be64  *desc;		/* KVA of descriptor ring */
 
-	volatile uint32_t state;
+	volatile int state;
 	struct adapter *adapter;
 	const __be64 *cdesc;	/* current descriptor */
 	uint8_t  gen;		/* generation bit */
 	uint8_t  intr_params;	/* interrupt holdoff parameters */
-	uint8_t  intr_next;	/* holdoff for next interrupt */
+	uint8_t  intr_next;	/* XXX: holdoff for next interrupt */
 	uint8_t  esize;		/* size (bytes) of each entry in the queue */
 	uint16_t qsize;		/* size (# of entries) of the queue */
 	uint16_t cidx;		/* consumer index */
-	uint16_t cntxt_id;	/* SGE context id  for the iq */
+	uint16_t cntxt_id;	/* SGE context id for the iq */
+
+	STAILQ_ENTRY(sge_iq) link;
 };
 
 enum {
+	EQ_CTRL		= 1,
+	EQ_ETH		= 2,
+#ifndef TCP_OFFLOAD_DISABLE
+	EQ_OFLD		= 3,
+#endif
+
 	/* eq flags */
-	EQ_ALLOCATED	= (1 << 1),	/* firmware resources allocated */
-	EQ_STARTED	= (1 << 2),	/* started */
-	EQ_CRFLUSHED	= (1 << 3),	/* expecting an update from SGE */
+	EQ_TYPEMASK	= 7,		/* 3 lsbits hold the type */
+	EQ_ALLOCATED	= (1 << 3),	/* firmware resources allocated */
+	EQ_DOOMED	= (1 << 4),	/* about to be destroyed */
+	EQ_CRFLUSHED	= (1 << 5),	/* expecting an update from SGE */
+	EQ_STALLED	= (1 << 6),	/* out of hw descriptors or dmamaps */
 };
 
 /*
@@ -291,10 +314,11 @@ enum {
  * consumes them) but it's special enough to have its own struct (see sge_fl).
  */
 struct sge_eq {
+	unsigned int flags;	/* MUST be first */
+	unsigned int cntxt_id;	/* SGE context id for the eq */
 	bus_dma_tag_t desc_tag;
 	bus_dmamap_t desc_map;
 	char lockname[16];
-	unsigned int flags;
 	struct mtx eq_lock;
 
 	struct tx_desc *desc;	/* KVA of descriptor ring */
@@ -307,9 +331,24 @@ struct sge_eq {
 	uint16_t pidx;		/* producer idx (desc idx) */
 	uint16_t pending;	/* # of descriptors used since last doorbell */
 	uint16_t iqid;		/* iq that gets egr_update for the eq */
-	unsigned int  cntxt_id;	/* SGE context id for the eq */
+	uint8_t tx_chan;	/* tx channel used by the eq */
+	struct task tx_task;
+	struct callout tx_callout;
+
+	/* stats */
+
+	uint32_t egr_update;	/* # of SGE_EGR_UPDATE notifications for eq */
+	uint32_t unstalled;	/* recovered from stall */
+};
+
+enum {
+	FL_STARVING	= (1 << 0), /* on the adapter's list of starving fl's */
+	FL_DOOMED	= (1 << 1), /* about to be destroyed */
 };
 
+#define FL_RUNNING_LOW(fl)	(fl->cap - fl->needed <= fl->lowat)
+#define FL_NOT_RUNNING_LOW(fl)	(fl->cap - fl->needed >= 2 * fl->lowat)
+
 struct sge_fl {
 	bus_dma_tag_t desc_tag;
 	bus_dmamap_t desc_map;
@@ -317,6 +356,7 @@ struct sge_fl {
 	uint8_t tag_idx;
 	struct mtx fl_lock;
 	char lockname[16];
+	int flags;
 
 	__be64 *desc;		/* KVA of descriptor ring, ptr to addresses */
 	bus_addr_t ba;		/* bus address of descriptor ring */
@@ -327,8 +367,10 @@ struct sge_fl {
 	uint32_t cidx;		/* consumer idx (buffer idx, NOT hw desc idx) */
 	uint32_t pidx;		/* producer idx (buffer idx, NOT hw desc idx) */
 	uint32_t needed;	/* # of buffers needed to fill up fl. */
+	uint32_t lowat;		/* # of buffers <= this means fl needs help */
 	uint32_t pending;	/* # of bufs allocated since last doorbell */
 	unsigned int dmamap_failed;
+	TAILQ_ENTRY(sge_fl) link; /* All starving freelists */
 };
 
 /* txq: SGE egress queue + what's needed for Ethernet NIC */
@@ -340,14 +382,8 @@ struct sge_txq {
 	struct buf_ring *br;	/* tx buffer ring */
 	struct tx_sdesc *sdesc;	/* KVA of software descriptor ring */
 	struct mbuf *m;		/* held up due to temporary resource shortage */
-	struct task resume_tx;
 
-	/* DMA maps used for tx */
-	struct tx_map *maps;
-	uint32_t map_total;	/* # of DMA maps */
-	uint32_t map_pidx;	/* next map to be used */
-	uint32_t map_cidx;	/* reclaimed up to this index */
-	uint32_t map_avail;	/* # of available maps */
+	struct tx_maps txmaps;
 
 	/* stats for common events first */
 
@@ -364,20 +400,14 @@ struct sge_txq {
 
 	uint32_t no_dmamap;	/* no DMA map to load the mbuf */
 	uint32_t no_desc;	/* out of hardware descriptors */
-	uint32_t egr_update;	/* # of SGE_EGR_UPDATE notifications for txq */
 } __aligned(CACHE_LINE_SIZE);
 
-enum {
-	RXQ_LRO_ENABLED	= (1 << 0)
-};
-
 /* rxq: SGE ingress queue + SGE free list + miscellaneous items */
 struct sge_rxq {
 	struct sge_iq iq;	/* MUST be first */
-	struct sge_fl fl;
+	struct sge_fl fl;	/* MUST follow iq */
 
 	struct ifnet *ifp;	/* the interface this rxq belongs to */
-	unsigned int flags;
 #ifdef INET
 	struct lro_ctrl lro;	/* LRO state */
 #endif
@@ -391,12 +421,28 @@ struct sge_rxq {
 
 } __aligned(CACHE_LINE_SIZE);
 
-/* ctrlq: SGE egress queue + stats for control queue */
-struct sge_ctrlq {
+#ifndef TCP_OFFLOAD_DISABLE
+/* ofld_rxq: SGE ingress queue + SGE free list + miscellaneous items */
+struct sge_ofld_rxq {
+	struct sge_iq iq;	/* MUST be first */
+	struct sge_fl fl;	/* MUST follow iq */
+} __aligned(CACHE_LINE_SIZE);
+#endif
+
+/*
+ * wrq: SGE egress queue that is given prebuilt work requests.  Both the control
+ * and offload tx queues are of this type.
+ */
+struct sge_wrq {
 	struct sge_eq eq;	/* MUST be first */
 
+	struct adapter *adapter;
+	struct mbuf *head;	/* held up due to lack of descriptors */
+	struct mbuf *tail;	/* valid only if head is valid */
+
 	/* stats for common events first */
 
+	uint64_t tx_wrs;	/* # of tx work requests */
 
 	/* stats for not-that-common events */
 
@@ -404,20 +450,28 @@ struct sge_ctrlq {
 } __aligned(CACHE_LINE_SIZE);
 
 struct sge {
-	uint16_t timer_val[SGE_NTIMERS];
-	uint8_t  counter_val[SGE_NCOUNTERS];
+	int timer_val[SGE_NTIMERS];
+	int counter_val[SGE_NCOUNTERS];
 	int fl_starve_threshold;
 
-	int nrxq;	/* total rx queues (all ports and the rest) */
-	int ntxq;	/* total tx queues (all ports and the rest) */
-	int niq;	/* total ingress queues */
-	int neq;	/* total egress queues */
+	int nrxq;	/* total # of Ethernet rx queues */
+	int ntxq;	/* total # of Ethernet tx tx queues */
+#ifndef TCP_OFFLOAD_DISABLE
+	int nofldrxq;	/* total # of TOE rx queues */
+	int nofldtxq;	/* total # of TOE tx queues */
+#endif
+	int niq;	/* total # of ingress queues */
+	int neq;	/* total # of egress queues */
 
 	struct sge_iq fwq;	/* Firmware event queue */
-	struct sge_ctrlq *ctrlq;/* Control queues */
-	struct sge_iq *intrq;	/* Interrupt queues */
+	struct sge_wrq mgmtq;	/* Management queue (control queue) */
+	struct sge_wrq *ctrlq;	/* Control queues */
 	struct sge_txq *txq;	/* NIC tx queues */
 	struct sge_rxq *rxq;	/* NIC rx queues */
+#ifndef TCP_OFFLOAD_DISABLE
+	struct sge_wrq *ofld_txq;	/* TOE tx queues */
+	struct sge_ofld_rxq *ofld_rxq;	/* TOE rx queues */
+#endif
 
 	uint16_t iq_start;
 	int eq_start;
@@ -425,7 +479,12 @@ struct sge {
 	struct sge_eq **eqmap;	/* eq->cntxt_id to eq mapping */
 };
 
+struct rss_header;
+typedef int (*cpl_handler_t)(struct sge_iq *, const struct rss_header *,
+    struct mbuf *);
+
 struct adapter {
+	SLIST_ENTRY(adapter) link;
 	device_t dev;
 	struct cdev *cdev;
 
@@ -454,27 +513,47 @@ struct adapter {
 
 	struct sge sge;
 
+	struct taskqueue *tq[NCHAN];	/* taskqueues that flush data out */
 	struct port_info *port[MAX_NPORTS];
 	uint8_t chan_map[NCHAN];
+	uint32_t filter_mode;
 
+#ifndef TCP_OFFLOAD_DISABLE
+	struct uld_softc tom;
+	struct tom_tunables tt;
+#endif
 	struct l2t_data *l2t;	/* L2 table */
 	struct tid_info tids;
 
-	int registered_device_map;
 	int open_device_map;
+#ifndef TCP_OFFLOAD_DISABLE
+	int offload_map;
+#endif
 	int flags;
 
 	char fw_version[32];
+	unsigned int cfcsum;
 	struct adapter_params params;
 	struct t4_virt_res vres;
 
-	struct sysctl_ctx_list ctx; /* from first_port_up to last_port_down */
-	struct sysctl_oid *oid_fwq;
-	struct sysctl_oid *oid_ctrlq;
-	struct sysctl_oid *oid_intrq;
+	uint16_t linkcaps;
+	uint16_t niccaps;
+	uint16_t toecaps;
+	uint16_t rdmacaps;
+	uint16_t iscsicaps;
+	uint16_t fcoecaps;
+
+	struct sysctl_ctx_list ctx; /* from adapter_full_init to full_uninit */
 
 	struct mtx sc_lock;
 	char lockname[16];
+
+	/* Starving free lists */
+	struct mtx sfl_lock;	/* same cache-line as sc_lock? but that's ok */
+	TAILQ_HEAD(, sge_fl) sfl;
+	struct callout sfl_callout;
+
+	cpl_handler_t cpl_handler[256] __aligned(CACHE_LINE_SIZE);
 };
 
 #define ADAPTER_LOCK(sc)		mtx_lock(&(sc)->sc_lock)
@@ -516,11 +595,15 @@ struct adapter {
 #define for_each_rxq(pi, iter, rxq) \
 	rxq = &pi->adapter->sge.rxq[pi->first_rxq]; \
 	for (iter = 0; iter < pi->nrxq; ++iter, ++rxq)
+#define for_each_ofld_txq(pi, iter, ofld_txq) \
+	ofld_txq = &pi->adapter->sge.ofld_txq[pi->first_ofld_txq]; \
+	for (iter = 0; iter < pi->nofldtxq; ++iter, ++ofld_txq)
+#define for_each_ofld_rxq(pi, iter, ofld_rxq) \
+	ofld_rxq = &pi->adapter->sge.ofld_rxq[pi->first_ofld_rxq]; \
+	for (iter = 0; iter < pi->nofldrxq; ++iter, ++ofld_rxq)
 
 /* One for errors, one for firmware events */
 #define T4_EXTRA_INTR 2
-#define NINTRQ(sc) ((sc)->intr_count > T4_EXTRA_INTR ? \
-    (sc)->intr_count - T4_EXTRA_INTR : 1)
 
 static inline uint32_t
 t4_read_reg(struct adapter *sc, uint32_t reg)
@@ -599,29 +682,52 @@ static inline bool is_10G_port(const str
 	return ((pi->link_cfg.supported & FW_PORT_CAP_SPEED_10G) != 0);
 }
 
+static inline int tx_resume_threshold(struct sge_eq *eq)
+{
+	return (eq->qsize / 4);
+}
+
 /* t4_main.c */
-void cxgbe_txq_start(void *, int);
+void t4_tx_task(void *, int);
+void t4_tx_callout(void *);
 int t4_os_find_pci_capability(struct adapter *, int);
 int t4_os_pci_save_state(struct adapter *);
 int t4_os_pci_restore_state(struct adapter *);
 void t4_os_portmod_changed(const struct adapter *, int);
 void t4_os_link_changed(struct adapter *, int, int);
+void t4_iterate(void (*)(struct adapter *, void *), void *);
+int t4_register_cpl_handler(struct adapter *, int, cpl_handler_t);
 
 /* t4_sge.c */
 void t4_sge_modload(void);
-void t4_sge_init(struct adapter *);
+int t4_sge_init(struct adapter *);
 int t4_create_dma_tag(struct adapter *);
 int t4_destroy_dma_tag(struct adapter *);
 int t4_setup_adapter_queues(struct adapter *);
 int t4_teardown_adapter_queues(struct adapter *);
-int t4_setup_eth_queues(struct port_info *);
-int t4_teardown_eth_queues(struct port_info *);
+int t4_setup_port_queues(struct port_info *);
+int t4_teardown_port_queues(struct port_info *);
+int t4_alloc_tx_maps(struct tx_maps *, bus_dma_tag_t, int, int);
+void t4_free_tx_maps(struct tx_maps *, bus_dma_tag_t);
 void t4_intr_all(void *);
 void t4_intr(void *);
 void t4_intr_err(void *);
 void t4_intr_evt(void *);
 int t4_mgmt_tx(struct adapter *, struct mbuf *);
+int t4_wrq_tx_locked(struct adapter *, struct sge_wrq *, struct mbuf *);
 int t4_eth_tx(struct ifnet *, struct sge_txq *, struct mbuf *);
 void t4_update_fl_bufsize(struct ifnet *);
+int can_resume_tx(struct sge_eq *);
+
+static inline int t4_wrq_tx(struct adapter *sc, struct sge_wrq *wrq, struct mbuf *m)
+{
+	int rc;
+
+	TXQ_LOCK(wrq);
+	rc = t4_wrq_tx_locked(sc, wrq, m);
+	TXQ_UNLOCK(wrq);
+	return (rc);
+}
+
 
 #endif

Modified: stable/8/sys/dev/cxgbe/common/common.h
==============================================================================
--- stable/8/sys/dev/cxgbe/common/common.h	Mon Feb  6 21:50:11 2012	(r231102)
+++ stable/8/sys/dev/cxgbe/common/common.h	Tue Feb  7 01:26:29 2012	(r231103)
@@ -42,6 +42,15 @@ enum {
 
 enum { MEM_EDC0, MEM_EDC1, MEM_MC };
 
+enum {
+	MEMWIN0_APERTURE = 2048,
+	MEMWIN0_BASE     = 0x1b800,
+	MEMWIN1_APERTURE = 32768,
+	MEMWIN1_BASE     = 0x28000,
+	MEMWIN2_APERTURE = 65536,
+	MEMWIN2_BASE     = 0x30000,
+};
+
 enum dev_master { MASTER_CANT, MASTER_MAY, MASTER_MUST };
 
 enum dev_state { DEV_STATE_UNINIT, DEV_STATE_INIT, DEV_STATE_ERR };
@@ -53,8 +62,8 @@ enum {
 };
 
 #define FW_VERSION_MAJOR 1
-#define FW_VERSION_MINOR 3
-#define FW_VERSION_MICRO 10
+#define FW_VERSION_MINOR 4
+#define FW_VERSION_MICRO 16
 
 struct port_stats {
 	u64 tx_octets;            /* total # of octets in good frames */
@@ -190,7 +199,6 @@ struct tp_proxy_stats {
 struct tp_cpl_stats {
 	u32 req[4];
 	u32 rsp[4];
-	u32 tx_err[4];
 };
 
 struct tp_rdma_stats {
@@ -214,9 +222,9 @@ struct vpd_params {
 };
 
 struct pci_params {
-	unsigned int  vpd_cap_addr;
-	unsigned char speed;
-	unsigned char width;
+	unsigned int vpd_cap_addr;
+	unsigned short speed;
+	unsigned short width;
 };
 
 /*
@@ -239,20 +247,20 @@ struct adapter_params {
 
 	unsigned int fw_vers;
 	unsigned int tp_vers;
-	u8 api_vers[7];
 
 	unsigned short mtus[NMTUS];
 	unsigned short a_wnd[NCCTRL_WIN];
 	unsigned short b_wnd[NCCTRL_WIN];
 
-	unsigned int mc_size;             /* MC memory size */
-	unsigned int nfilters;            /* size of filter region */
+	unsigned int mc_size;		/* MC memory size */
+	unsigned int nfilters;		/* size of filter region */
 
 	unsigned int cim_la_size;
 
-	unsigned int nports;             /* # of ethernet ports */
+	/* Used as int in sysctls, do not reduce size */
+	unsigned int nports;		/* # of ethernet ports */
 	unsigned int portvec;
-	unsigned int rev;                /* chip revision */
+	unsigned int rev;		/* chip revision */
 	unsigned int offload;
 
 	unsigned int ofldq_wr_cred;
@@ -366,6 +374,9 @@ int t4_seeprom_wp(struct adapter *adapte
 int t4_read_flash(struct adapter *adapter, unsigned int addr, unsigned int nwords,
 		  u32 *data, int byte_oriented);
 int t4_load_fw(struct adapter *adapter, const u8 *fw_data, unsigned int size);
+int t4_load_boot(struct adapter *adap, const u8 *boot_data,
+                 unsigned int boot_addr, unsigned int size);
+unsigned int t4_flash_cfg_addr(struct adapter *adapter);
 int t4_load_cfg(struct adapter *adapter, const u8 *cfg_data, unsigned int size);
 int t4_get_fw_version(struct adapter *adapter, u32 *vers);
 int t4_get_tp_version(struct adapter *adapter, u32 *vers);
@@ -460,8 +471,8 @@ int t4_wol_pat_enable(struct adapter *ad
 int t4_fw_hello(struct adapter *adap, unsigned int mbox, unsigned int evt_mbox,
 		enum dev_master master, enum dev_state *state);
 int t4_fw_bye(struct adapter *adap, unsigned int mbox);
-int t4_early_init(struct adapter *adap, unsigned int mbox);
 int t4_fw_reset(struct adapter *adap, unsigned int mbox, int reset);
+int t4_fw_initialize(struct adapter *adap, unsigned int mbox);
 int t4_query_params(struct adapter *adap, unsigned int mbox, unsigned int pf,
 		    unsigned int vf, unsigned int nparams, const u32 *params,
 		    u32 *val);

Modified: stable/8/sys/dev/cxgbe/common/t4_hw.c
==============================================================================
--- stable/8/sys/dev/cxgbe/common/t4_hw.c	Mon Feb  6 21:50:11 2012	(r231102)
+++ stable/8/sys/dev/cxgbe/common/t4_hw.c	Tue Feb  7 01:26:29 2012	(r231103)
@@ -30,10 +30,10 @@ __FBSDID("$FreeBSD$");
 #include "common.h"
 #include "t4_regs.h"
 #include "t4_regs_values.h"
-#include "t4fw_interface.h"
+#include "firmware/t4fw_interface.h"
 
 #undef msleep
-#define msleep(x) DELAY((x) * 1000)
+#define msleep(x) pause("t4hw", (x) * hz / 1000)
 
 /**
  *	t4_wait_op_done_val - wait until an operation is completed
@@ -187,7 +187,7 @@ int t4_wr_mbox_meat(struct adapter *adap
 	 * off to larger delays to a maximum retry delay.
 	 */
 	static const int delay[] = {
-		1, 1, 3, 5, 10, 10, 20, 50, 100, 200
+		1, 1, 3, 5, 10, 10, 20, 50, 100
 	};
 
 	u32 v;
@@ -625,17 +625,6 @@ enum {
 	SF_RD_DATA_FAST = 0xb,        /* read flash */
 	SF_RD_ID        = 0x9f,       /* read ID */
 	SF_ERASE_SECTOR = 0xd8,       /* erase sector */
-
-	FW_START_SEC = 8,             /* first flash sector for FW */
-	FW_END_SEC = 15,              /* last flash sector for FW */
-	FW_IMG_START = FW_START_SEC * SF_SEC_SIZE,
-	FW_MAX_SIZE = (FW_END_SEC - FW_START_SEC + 1) * SF_SEC_SIZE,
-        
-	FLASH_CFG_MAX_SIZE    = 0x10000 , /* max size of the flash config file */
-	FLASH_CFG_OFFSET      = 0x1f0000,
-	FLASH_CFG_START_SEC   = FLASH_CFG_OFFSET / SF_SEC_SIZE,
-	FPGA_FLASH_CFG_OFFSET = 0xf0000 , /* if FPGA mode, then cfg file is at 1MB - 64KB */
-	FPGA_FLASH_CFG_START_SEC  = FPGA_FLASH_CFG_OFFSET / SF_SEC_SIZE,
 };
 
 /**
@@ -763,12 +752,15 @@ int t4_read_flash(struct adapter *adapte
  *	@addr: the start address to write
  *	@n: length of data to write in bytes
  *	@data: the data to write
+ *	@byte_oriented: whether to store data as bytes or as words
  *
  *	Writes up to a page of data (256 bytes) to the serial flash starting
  *	at the given address.  All the data must be written to the same page.
+ *	If @byte_oriented is set the write data is stored as byte stream 
+ *	(i.e. matches what on disk), otherwise in big-endian.
  */
 static int t4_write_flash(struct adapter *adapter, unsigned int addr,
-			  unsigned int n, const u8 *data)
+			  unsigned int n, const u8 *data, int byte_oriented)
 {
 	int ret;
 	u32 buf[SF_PAGE_SIZE / 4];
@@ -788,6 +780,9 @@ static int t4_write_flash(struct adapter
 		for (val = 0, i = 0; i < c; ++i)
 			val = (val << 8) + *data++;
 
+		if (!byte_oriented)
+			val = htonl(val);
+
 		ret = sf1_write(adapter, c, c != left, 1, val);
 		if (ret)
 			goto unlock;
@@ -799,7 +794,8 @@ static int t4_write_flash(struct adapter
 	t4_write_reg(adapter, A_SF_OP, 0);    /* unlock SF */
 
 	/* Read the page to verify the write succeeded */
-	ret = t4_read_flash(adapter, addr & ~0xff, ARRAY_SIZE(buf), buf, 1);
+	ret = t4_read_flash(adapter, addr & ~0xff, ARRAY_SIZE(buf), buf,
+			    byte_oriented);
 	if (ret)
 		return ret;
 
@@ -825,7 +821,7 @@ unlock:
 int t4_get_fw_version(struct adapter *adapter, u32 *vers)
 {
 	return t4_read_flash(adapter,
-			     FW_IMG_START + offsetof(struct fw_hdr, fw_ver), 1,
+			     FLASH_FW_START + offsetof(struct fw_hdr, fw_ver), 1,
 			     vers, 0);
 }
 
@@ -838,7 +834,7 @@ int t4_get_fw_version(struct adapter *ad
  */
 int t4_get_tp_version(struct adapter *adapter, u32 *vers)
 {
-	return t4_read_flash(adapter, FW_IMG_START + offsetof(struct fw_hdr,
+	return t4_read_flash(adapter, FLASH_FW_START + offsetof(struct fw_hdr,
 							      tp_microcode_ver),
 			     1, vers, 0);
 }
@@ -854,24 +850,17 @@ int t4_get_tp_version(struct adapter *ad
  */
 int t4_check_fw_version(struct adapter *adapter)
 {
-	u32 api_vers[2];
 	int ret, major, minor, micro;
 
 	ret = t4_get_fw_version(adapter, &adapter->params.fw_vers);
 	if (!ret)
 		ret = t4_get_tp_version(adapter, &adapter->params.tp_vers);
-	if (!ret)
-		ret = t4_read_flash(adapter,
-			FW_IMG_START + offsetof(struct fw_hdr, intfver_nic),
-			2, api_vers, 1);
 	if (ret)
 		return ret;
 
 	major = G_FW_HDR_FW_VER_MAJOR(adapter->params.fw_vers);
 	minor = G_FW_HDR_FW_VER_MINOR(adapter->params.fw_vers);
 	micro = G_FW_HDR_FW_VER_MICRO(adapter->params.fw_vers);
-	memcpy(adapter->params.api_vers, api_vers,
-	       sizeof(adapter->params.api_vers));
 
 	if (major != FW_VERSION_MAJOR) {            /* major mismatch - fail */
 		CH_ERR(adapter, "card FW has major version %u, driver wants "
@@ -914,6 +903,21 @@ static int t4_flash_erase_sectors(struct
 }
 
 /**
+ *	t4_flash_cfg_addr - return the address of the flash configuration file
+ *	@adapter: the adapter
+ *
+ *	Return the address within the flash where the Firmware Configuration
+ *	File is stored.
+ */
+unsigned int t4_flash_cfg_addr(struct adapter *adapter)
+{
+	if (adapter->params.sf_size == 0x100000)
+		return FLASH_FPGA_CFG_START;
+	else
+		return FLASH_CFG_START;
+}
+
+/**
  *	t4_load_cfg - download config file
  *	@adap: the adapter
  *	@cfg_data: the cfg text file to write
@@ -928,17 +932,8 @@ int t4_load_cfg(struct adapter *adap, co
 	unsigned int flash_cfg_start_sec;
 	unsigned int sf_sec_size = adap->params.sf_size / adap->params.sf_nsec;
 
-	if (adap->params.sf_size == 0x100000) {
-		addr = FPGA_FLASH_CFG_OFFSET;
-		flash_cfg_start_sec = FPGA_FLASH_CFG_START_SEC;
-	} else {
-		addr = FLASH_CFG_OFFSET;
-		flash_cfg_start_sec = FLASH_CFG_START_SEC;
-	}
-	if (!size) {
-		CH_ERR(adap, "cfg file has no data\n");
-		return -EINVAL;
-	}
+	addr = t4_flash_cfg_addr(adap);
+	flash_cfg_start_sec = addr / SF_SEC_SIZE;
 
 	if (size > FLASH_CFG_MAX_SIZE) {
 		CH_ERR(adap, "cfg file too large, max is %u bytes\n",
@@ -950,7 +945,11 @@ int t4_load_cfg(struct adapter *adap, co
 			 sf_sec_size);
 	ret = t4_flash_erase_sectors(adap, flash_cfg_start_sec,
 				     flash_cfg_start_sec + i - 1);
-	if (ret)
+	/*
+	 * If size == 0 then we're simply erasing the FLASH sectors associated
+	 * with the on-adapter Firmware Configuration File.
+	 */
+	if (ret || size == 0)
 		goto out;
 
         /* this will write to the flash up to SF_PAGE_SIZE at a time */
@@ -959,7 +958,7 @@ int t4_load_cfg(struct adapter *adap, co
 			n = size - i;
 		else 
 			n = SF_PAGE_SIZE;
-		ret = t4_write_flash(adap, addr, n, cfg_data);
+		ret = t4_write_flash(adap, addr, n, cfg_data, 1);
 		if (ret)
 			goto out;
 		
@@ -969,7 +968,8 @@ int t4_load_cfg(struct adapter *adap, co
                 
 out:
 	if (ret)
-		CH_ERR(adap, "config file download failed %d\n", ret);
+		CH_ERR(adap, "config file %s failed %d\n",
+		       (size == 0 ? "clear" : "download"), ret);
 	return ret;
 }
 
@@ -1004,9 +1004,9 @@ int t4_load_fw(struct adapter *adap, con
 		CH_ERR(adap, "FW image size differs from size in FW header\n");
 		return -EINVAL;
 	}
-	if (size > FW_MAX_SIZE) {
+	if (size > FLASH_FW_MAX_SIZE) {
 		CH_ERR(adap, "FW image too large, max is %u bytes\n",
-		       FW_MAX_SIZE);
+		       FLASH_FW_MAX_SIZE);
 		return -EFBIG;
 	}
 
@@ -1020,7 +1020,8 @@ int t4_load_fw(struct adapter *adap, con
 	}
 
 	i = DIV_ROUND_UP(size, sf_sec_size);        /* # of sectors spanned */
-	ret = t4_flash_erase_sectors(adap, FW_START_SEC, FW_START_SEC + i - 1);
+	ret = t4_flash_erase_sectors(adap, FLASH_FW_START_SEC,
+	    FLASH_FW_START_SEC + i - 1);
 	if (ret)
 		goto out;
 
@@ -1031,28 +1032,110 @@ int t4_load_fw(struct adapter *adap, con
 	 */
 	memcpy(first_page, fw_data, SF_PAGE_SIZE);
 	((struct fw_hdr *)first_page)->fw_ver = htonl(0xffffffff);
-	ret = t4_write_flash(adap, FW_IMG_START, SF_PAGE_SIZE, first_page);
+	ret = t4_write_flash(adap, FLASH_FW_START, SF_PAGE_SIZE, first_page, 1);
 	if (ret)
 		goto out;
 
-	addr = FW_IMG_START;
+	addr = FLASH_FW_START;
 	for (size -= SF_PAGE_SIZE; size; size -= SF_PAGE_SIZE) {
 		addr += SF_PAGE_SIZE;
 		fw_data += SF_PAGE_SIZE;
-		ret = t4_write_flash(adap, addr, SF_PAGE_SIZE, fw_data);
+		ret = t4_write_flash(adap, addr, SF_PAGE_SIZE, fw_data, 1);
 		if (ret)
 			goto out;
 	}
 
 	ret = t4_write_flash(adap,
-			     FW_IMG_START + offsetof(struct fw_hdr, fw_ver),
-			     sizeof(hdr->fw_ver), (const u8 *)&hdr->fw_ver);
+			     FLASH_FW_START + offsetof(struct fw_hdr, fw_ver),
+			     sizeof(hdr->fw_ver), (const u8 *)&hdr->fw_ver, 1);
 out:
 	if (ret)
 		CH_ERR(adap, "firmware download failed, error %d\n", ret);
 	return ret;
 }
 
+/* BIOS boot header */
+typedef struct boot_header_s {
+	u8	signature[2];	/* signature */
+	u8	length;		/* image length (include header) */
+	u8	offset[4];	/* initialization vector */
+	u8	reserved[19];	/* reserved */
+	u8	exheader[2];	/* offset to expansion header */
+} boot_header_t;
+
+enum {
+	BOOT_FLASH_BOOT_ADDR = 0x0,/* start address of boot image in flash */
+	BOOT_SIGNATURE = 0xaa55,   /* signature of BIOS boot ROM */
+	BOOT_SIZE_INC = 512,       /* image size measured in 512B chunks */
+	BOOT_MIN_SIZE = sizeof(boot_header_t), /* at least basic header */
+	BOOT_MAX_SIZE = 1024*BOOT_SIZE_INC /* 1 byte * length increment  */
+};
+
+/*
+ *	t4_load_boot - download boot flash
+ *	@adapter: the adapter
+ *	@boot_data: the boot image to write
+ *	@size: image size
+ *
+ *	Write the supplied boot image to the card's serial flash.
+ *	The boot image has the following sections: a 28-byte header and the
+ *	boot image.
+ */
+int t4_load_boot(struct adapter *adap, const u8 *boot_data, 
+		 unsigned int boot_addr, unsigned int size)
+{
+	int ret, addr;
+	unsigned int i;
+	unsigned int boot_sector = boot_addr * 1024;
+	unsigned int sf_sec_size = adap->params.sf_size / adap->params.sf_nsec;
+
+	/*
+	 * Perform some primitive sanity testing to avoid accidentally
+	 * writing garbage over the boot sectors.  We ought to check for
+	 * more but it's not worth it for now ...
+	 */
+	if (size < BOOT_MIN_SIZE || size > BOOT_MAX_SIZE) {
+		CH_ERR(adap, "boot image too small/large\n");
+		return -EFBIG;
+	}
+
+	/*
+	 * Make sure the boot image does not encroach on the firmware region
+	 */
+	if ((boot_sector + size) >> 16 > FLASH_FW_START_SEC) {
+		CH_ERR(adap, "boot image encroaching on firmware region\n");
+		return -EFBIG;
+	}
+
+	i = DIV_ROUND_UP(size, sf_sec_size);        /* # of sectors spanned */
+	ret = t4_flash_erase_sectors(adap, boot_sector >> 16, 
+				     (boot_sector >> 16) + i - 1);

*** DIFF OUTPUT TRUNCATED AT 1000 LINES ***



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201202070126.q171QT7C062353>