From owner-freebsd-net@FreeBSD.ORG Mon Apr 12 19:42:19 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 644C21065672 for ; Mon, 12 Apr 2010 19:42:19 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-bw0-f216.google.com (mail-bw0-f216.google.com [209.85.218.216]) by mx1.freebsd.org (Postfix) with ESMTP id CC6998FC0A for ; Mon, 12 Apr 2010 19:42:17 +0000 (UTC) Received: by bwz8 with SMTP id 8so3425860bwz.3 for ; Mon, 12 Apr 2010 12:42:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:date:to:cc :subject:message-id:reply-to:references:mime-version:content-type :content-disposition:in-reply-to:user-agent; bh=8eeIJEDtuoFa4RZ84A25HqgF1ZXOeY8m7lIh+Hch/yA=; b=nIVcL/YXTBqCRnU33vj03zP2eV+HEKgXWsxf0V+kzif45lZBo8UyOTTWDql0mYTHZr yuhymrLNCuxWQxcbQrFmNncvTXJsLWQtjKVxDOAlVx1hRRqZ2RpxMnOtQyJG+6kHtB6C JeoAFfyYGPXnMXe5vMkBtc0s6VIk/MLjOylgU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=iT1hVlKW9pLCkBG5u9kuO3ohCEnFu0x19K2HDby0pULvAc/l5q2teO9XnPfy7MX4UZ bfCaojm9zjlN8qavE6urEbr0p8+YtkanG2UwJRQreW6Iflrb01mu0vkimEqpb/NBWi3K RJMDlc7Vc0TISncbdYx01nvP7QBVyL2khsuVE= Received: by 10.204.39.203 with SMTP id h11mr5481079bke.10.1271101336989; Mon, 12 Apr 2010 12:42:16 -0700 (PDT) Received: from pyunyh@gmail.com ([174.35.1.224]) by mx.google.com with ESMTPS id 14sm1956989bwz.2.2010.04.12.12.42.12 (version=TLSv1/SSLv3 cipher=RC4-MD5); Mon, 12 Apr 2010 12:42:15 -0700 (PDT) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Mon, 12 Apr 2010 12:42:09 -0700 From: Pyun YongHyeon Date: Mon, 12 Apr 2010 12:42:09 -0700 To: "Erich Jenkins, Fuujin Group Ltd" Message-ID: <20100412194209.GF1444@michelle.cdnetworks.com> References: <4BBED9A4.3080303@fuujingroup.com> <20100409070147.GA77350@korolev-net.ru> <4BBEE18C.6040204@fuujingroup.com> <20100409173821.GD1085@michelle.cdnetworks.com> <4BC016F3.4020300@fuujingroup.com> <20100410212520.GB6481@michelle.cdnetworks.com> <4BC12097.4030508@fuujingroup.com> <4BC19324.3050800@fuujingroup.com> <20100412175701.GC1444@michelle.cdnetworks.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="wRRV7LY7NUeQGEoC" Content-Disposition: inline In-Reply-To: <20100412175701.GC1444@michelle.cdnetworks.com> User-Agent: Mutt/1.4.2.3i Cc: freebsd-net@freebsd.org, Evgenii Davidov Subject: Re: Broadcom BCM5701 / HP NC6770 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Apr 2010 19:42:19 -0000 --wRRV7LY7NUeQGEoC Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Mon, Apr 12, 2010 at 10:57:01AM -0700, Pyun YongHyeon wrote: > On Sun, Apr 11, 2010 at 03:15:16AM -0600, Erich Jenkins, Fuujin Group Ltd wrote: > > I've been muddling around in src/sys/dev on the old system and the new > > system and there appear to be rather major changes to MII and bge, > > possibly the whole stack? > > > > It was not completely rewritten but many improvements were made. > > > There are a number of things that seem to have been merged with other > > parts of the network stack, or perhaps written into the individual > > drivers (someone working on the net stack would have to verify that). > > > > For instance, some files called in 5.3-REL seem to have gone away > > completely, and in the new (unpatched) version of if_bge.c under > > 7.3-REL, calls to these modules are gone: > > > > - #include /* for vtophys */ > > - #include /* for vtophys */ > > One of the most significant changes would be bus_dma(9) conversion > which is required to all drivers to make it work correctly on a > variety of platforms. bus_dma(9) does not directly use vtophys > anymore so these headers were nuked. > > > - #include /* for DELAY */ > > - #include > > > > - #include (called but something changed in here) > > - #include (ditto above) > > > > No, these headers are still present. > > > It appears that the checksum features have been completely rewritten, > > Checksum offloading was not completely rewritten but workaround > for buggy controllers was added. > > > and some of the ring settings have changed. It's interesting that the > > driver only fills 256 of the rx rings in the hopes that the cpu is "fast > > enough to keep up with the NIC". Would a subroutine here to grab the cpu > > That magic number 256 is adequate for most cases but it may not be > enough to handle heavy loads. Internally the controller use fixed > 512 RX buffers but bge(4) used only half of the buffers to save > resources. I think you can increase SSLOTS to 512 to get full 512 > RX buffers. > > > clock and count (number of procs/pipelines) be more trouble than it's > > worth to "automagically" increase the number of rx rings the driver > > fills based on the system in which it's installed? > > > > Dynamically increasing number of RX buffers is doable but it would > add much more code. If there is high demand for that I would just > increase number of RX buffers to 512. Controller can't be > configured to have more than 512 RX buffers. > > > Something also changed in pci/pcireg.h and pci/pcivar.h, but I haven't > > had the time to hunt down and expand the source tree from the 5.3-REL > > branch yet. > > > > I have other machines with copper nics utilizing the bge driver, and > > there are no issues at all. Perhaps I'm getting ahead of things, but > > Yes that is expected one. :-) > > > since this seems to have been broken through several releases, would it > > make any sense to split the support between the BCM5701KHB chipset and > > the more recent BCM chipset to avoid causing issues with cards/systems > > not currently experiencing troubles? > > > > I'd like to if I can. Supporting huge number of different > controllers in single driver is maintenance nightmare. However, > rewriting some part that require special handling for certain > controller/revision is too risky because I don't have access to > most controllers. > > One theory for the issue I got while reading the code is link state > handling. As I said in previous mail, link state handling for TBI > is somewhat tricky in bge(4) and driver seemed to rely on periodic > register access to keep track of link state. I guess polling(4) may > give different behavior on link state handling as it does not rely > on interrupts at all. So would you try to use polling(4) and see > that make any difference on your box? If polling(4) make it work, try attached patch. --wRRV7LY7NUeQGEoC Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="bge.5701.diff2" Index: sys/dev/bge/if_bge.c =================================================================== --- sys/dev/bge/if_bge.c (revision 206498) +++ sys/dev/bge/if_bge.c (working copy) @@ -3711,6 +3711,7 @@ { struct bge_softc *sc = xsc; struct mii_data *mii = NULL; + int armintr; BGE_LOCK_ASSERT(sc); @@ -3739,18 +3740,16 @@ * link status manually. Here we register pending link event * and trigger interrupt. */ + #ifdef DEVICE_POLLING /* In polling mode we poll link state in bge_poll(). */ - if (!(sc->bge_ifp->if_capenable & IFCAP_POLLING)) + armintr = (sc->bge_ifp->if_capenable & IFCAP_POLLING) == 0; +#else + armintr = 1; #endif - { sc->bge_link_evt++; - if (sc->bge_asicrev == BGE_ASICREV_BCM5700 || - sc->bge_flags & BGE_FLAG_5788) + if (armintr != 0) BGE_SETBIT(sc, BGE_MISC_LOCAL_CTL, BGE_MLC_INTR_SET); - else - BGE_SETBIT(sc, BGE_HCC_MODE, BGE_HCCMODE_COAL_NOW); - } } bge_asf_driver_up(sc); --wRRV7LY7NUeQGEoC--