From owner-freebsd-net@FreeBSD.ORG Thu Mar 8 02:06:42 2012 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 73484106564A; Thu, 8 Mar 2012 02:06:42 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-pz0-f54.google.com (mail-pz0-f54.google.com [209.85.210.54]) by mx1.freebsd.org (Postfix) with ESMTP id 3222A8FC12; Thu, 8 Mar 2012 02:06:41 +0000 (UTC) Received: by dald2 with SMTP id d2so3630dal.13 for ; Wed, 07 Mar 2012 18:06:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=5SCmV/iV4N5w3eUp5QHz15kUI4K8ZzMaVscMn/DEMCA=; b=rggQ+5eMFMj/q4N4r/33xOmAMq7tP5ApU+1Whq0CWylGvwqWMIRgZ5/GdCTgspMPCl dUgVQPUBfnuJT1sTpfLPsaPj4j1QM7ZA2R+lP5UUY1J1di8i/QW+pijsS2xp9QbXHyYm AAhOLTEXctkbrordPfJRGxmY4eA/UiY53w6js1HsHBTjIxGhvKfjbQPaQ5TmUauJVB46 r+kj6yKEv1NzuOmkOQ2OGtqi5is80WZn0YEBMkeOshx9t4NE9T7XCQtbZEKEPeN0Pjcy r6HAw/vgjJhaxQqN5vTKRqCgsMtdD3E4CoqxWGPk7MnwJG4O0i9RuBtwcGaUMR981ZE3 i5ZQ== Received: by 10.68.233.98 with SMTP id tv2mr6732430pbc.51.1331172401682; Wed, 07 Mar 2012 18:06:41 -0800 (PST) Received: from pyunyh@gmail.com ([114.111.62.249]) by mx.google.com with ESMTPS id y9sm1752930pbu.40.2012.03.07.18.06.36 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 07 Mar 2012 18:06:38 -0800 (PST) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Thu, 08 Mar 2012 11:06:28 -0800 From: YongHyeon PYUN Date: Thu, 8 Mar 2012 11:06:28 -0800 To: Eugene Grosbein Message-ID: <20120308190628.GB13138@michelle.cdnetworks.com> References: <4F5608EA.6080705@rdtc.ru> <20120307202914.GB9436@michelle.cdnetworks.com> <4F571870.3090902@rdtc.ru> <20120308034345.GD9436@michelle.cdnetworks.com> <4F578FE1.1000808@rdtc.ru> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="EVF5PPMfhYS0aIcm" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <4F578FE1.1000808@rdtc.ru> User-Agent: Mutt/1.4.2.3i Cc: marius@freebsd.org, "net@freebsd.org" , yongari@freebsd.org Subject: Re: suboptimal bge(4) BCM5704 performance in RELENG_8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Mar 2012 02:06:42 -0000 --EVF5PPMfhYS0aIcm Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit On Wed, Mar 07, 2012 at 11:42:09PM +0700, Eugene Grosbein wrote: > 08.03.2012 10:43, YongHyeon PYUN пишет: > > >>> Show me the output of "sysctl dev.bge.0.stats". > >> > >> # sysctl dev.bge.0.stats > >> dev.bge.0.stats.FramesDroppedDueToFilters: 0 > >> dev.bge.0.stats.DmaWriteQueueFull: 84072 > >> dev.bge.0.stats.DmaWriteHighPriQueueFull: 0 > >> dev.bge.0.stats.NoMoreRxBDs: 0 > >> dev.bge.0.stats.InputDiscards: 0 > >> dev.bge.0.stats.InputErrors: 30 > >> dev.bge.0.stats.RecvThresholdHit: 745400662 > >> dev.bge.0.stats.DmaReadQueueFull: 2020586592 > >> dev.bge.0.stats.DmaReadHighPriQueueFull: 0 > >> dev.bge.0.stats.SendDataCompQueueFull: 0 > >> dev.bge.0.stats.RingSetSendProdIndex: 2832885493 > >> dev.bge.0.stats.RingStatusUpdate: 899990835 > >> dev.bge.0.stats.Interrupts: 899990835 > >> dev.bge.0.stats.AvoidedInterrupts: 0 > >> dev.bge.0.stats.SendThresholdHit: 0 > >> dev.bge.0.stats.rx.ifHCInOctets: 491268800 > >> dev.bge.0.stats.rx.Fragments: 234 > >> dev.bge.0.stats.rx.UnicastPkts: 1977202324 > >> dev.bge.0.stats.rx.MulticastPkts: 0 > >> dev.bge.0.stats.rx.FCSErrors: 341 > > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > You have multiple FCS and Input errors. Check signal > > quality(i.e. UTP cable). > > OTOH, should not bge(4) for such errors increase "Ierrs" counters > that shows netstat -i? > You're right. I checked the code and noticed that old controllers(BCM570[0-4]) do not report these errors. Would you try attached patch and let me know whether it corrects statistics counter? The patch also will show more information like whether the controller runs in PCI mode or not as well as showing frequency of PCI clock speed. > Eugene Grosbein --EVF5PPMfhYS0aIcm Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="bge.570x.stats.diff" Index: sys/dev/bge/if_bgereg.h =================================================================== --- sys/dev/bge/if_bgereg.h (revision 232655) +++ sys/dev/bge/if_bgereg.h (working copy) @@ -1989,7 +1989,9 @@ /* Misc. config register */ #define BGE_MISCCFG_RESET_CORE_CLOCKS 0x00000001 #define BGE_MISCCFG_TIMER_PRESCALER 0x000000FE -#define BGE_MISCCFG_BOARD_ID 0x0001E000 +#define BGE_MISCCFG_BOARD_ID_MASK 0x0001E000 +#define BGE_MISCCFG_BOARD_ID_5704 0x00000000 +#define BGE_MISCCFG_BOARD_ID_5704CIOBE 0x00004000 #define BGE_MISCCFG_BOARD_ID_5788 0x00010000 #define BGE_MISCCFG_BOARD_ID_5788M 0x00018000 #define BGE_MISCCFG_EPHY_IDDQ 0x00200000 @@ -2868,6 +2870,8 @@ struct bge_softc { int bge_csum_features; struct callout bge_stat_ch; uint32_t bge_rx_discards; + uint32_t bge_rx_inerrs; + uint32_t bge_rx_nobds; uint32_t bge_tx_discards; uint32_t bge_tx_collisions; #ifdef DEVICE_POLLING Index: sys/dev/bge/if_bge.c =================================================================== --- sys/dev/bge/if_bge.c (revision 232655) +++ sys/dev/bge/if_bge.c (working copy) @@ -380,6 +380,8 @@ static void bge_dma_free(struct bge_softc *); static int bge_dma_ring_alloc(struct bge_softc *, bus_size_t, bus_size_t, bus_dma_tag_t *, uint8_t **, bus_dmamap_t *, bus_addr_t *, const char *); +static void bge_devinfo(struct bge_softc *); + static int bge_get_eaddr_fw(struct bge_softc *sc, uint8_t ether_addr[]); static int bge_get_eaddr_mem(struct bge_softc *, uint8_t[]); static int bge_get_eaddr_nvram(struct bge_softc *, uint8_t[]); @@ -2774,6 +2776,59 @@ bge_can_use_msi(struct bge_softc *sc) return (can_use_msi); } +static void +bge_devinfo(struct bge_softc *sc) +{ + uint32_t cfg, clk; + + device_printf(sc->bge_dev, + "CHIP ID 0x%08x; ASIC REV 0x%02x; CHIP REV 0x%02x; ", + sc->bge_chipid, sc->bge_asicrev, sc->bge_chiprev); + if (sc->bge_flags & BGE_FLAG_PCIE) + printf("PCI-E\n"); + else if (sc->bge_flags & BGE_FLAG_PCIX) { + printf("PCI-X "); + cfg = CSR_READ_4(sc, BGE_MISC_CFG) & BGE_MISCCFG_BOARD_ID_MASK; + if (cfg == BGE_MISCCFG_BOARD_ID_5704CIOBE) + clk = 133; + else { + clk = CSR_READ_4(sc, BGE_PCI_CLKCTL) & 0x1F; + switch (clk) { + case 0: + clk = 33; + break; + case 2: + clk = 50; + break; + case 4: + clk = 66; + break; + case 6: + clk = 100; + break; + case 7: + clk = 133; + break; + } + } + printf("%u MHz\n", clk); + } else { + if (sc->bge_pcixcap != 0) + printf("PCI on PCI-X "); + else + printf("PCI "); + cfg = pci_read_config(sc->bge_dev, BGE_PCI_PCISTATE, 4); + if (cfg & BGE_PCISTATE_PCI_BUSSPEED) + clk = 66; + else + clk = 33; + if (cfg & BGE_PCISTATE_32BIT_BUS) + printf("%u MHz; 32bit\n", clk); + else + printf("%u MHz; 64bit\n", clk); + } +} + static int bge_attach(device_t dev) { @@ -3002,7 +3057,7 @@ bge_attach(device_t dev) if (sc->bge_asicrev == BGE_ASICREV_BCM5719) sc->bge_flags |= BGE_FLAG_4K_RDMA_BUG; - misccfg = CSR_READ_4(sc, BGE_MISC_CFG) & BGE_MISCCFG_BOARD_ID; + misccfg = CSR_READ_4(sc, BGE_MISC_CFG) & BGE_MISCCFG_BOARD_ID_MASK; if (sc->bge_asicrev == BGE_ASICREV_BCM5705) { if (misccfg == BGE_MISCCFG_BOARD_ID_5788 || misccfg == BGE_MISCCFG_BOARD_ID_5788M) @@ -3132,11 +3187,7 @@ bge_attach(device_t dev) goto fail; } - device_printf(dev, - "CHIP ID 0x%08x; ASIC REV 0x%02x; CHIP REV 0x%02x; %s\n", - sc->bge_chipid, sc->bge_asicrev, sc->bge_chiprev, - (sc->bge_flags & BGE_FLAG_PCIX) ? "PCI-X" : - ((sc->bge_flags & BGE_FLAG_PCIE) ? "PCI-E" : "PCI")); + bge_devinfo(sc); BGE_LOCK_INIT(sc, device_get_nameunit(dev)); @@ -4400,6 +4451,12 @@ bge_stats_update(struct bge_softc *sc) ifp->if_collisions += (uint32_t)(cnt - sc->bge_tx_collisions); sc->bge_tx_collisions = cnt; + cnt = READ_STAT(sc, stats, nicNoMoreRxBDs.bge_addr_lo); + ifp->if_ierrors += (uint32_t)(cnt - sc->bge_rx_nobds); + sc->bge_rx_nobds = cnt; + cnt = READ_STAT(sc, stats, ifInErrors.bge_addr_lo); + ifp->if_ierrors += (uint32_t)(cnt - sc->bge_rx_inerrs); + sc->bge_rx_inerrs = cnt; cnt = READ_STAT(sc, stats, ifInDiscards.bge_addr_lo); ifp->if_ierrors += (uint32_t)(cnt - sc->bge_rx_discards); sc->bge_rx_discards = cnt; --EVF5PPMfhYS0aIcm--