From owner-svn-src-all@FreeBSD.ORG Wed Oct 15 19:00:27 2008 Return-Path: Delivered-To: svn-src-all@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 12997106564A; Wed, 15 Oct 2008 19:00:27 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from fallbackmx10.syd.optusnet.com.au (fallbackmx10.syd.optusnet.com.au [211.29.132.251]) by mx1.freebsd.org (Postfix) with ESMTP id 00C638FC19; Wed, 15 Oct 2008 19:00:25 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail05.syd.optusnet.com.au (mail05.syd.optusnet.com.au [211.29.132.186]) by fallbackmx10.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id m9F9fmx8024212; Wed, 15 Oct 2008 20:41:49 +1100 Received: from c211-30-84-14.carlnfd3.nsw.optusnet.com.au (c211-30-84-14.carlnfd3.nsw.optusnet.com.au [211.30.84.14]) by mail05.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id m9F9femV032073 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 15 Oct 2008 20:41:43 +1100 Date: Wed, 15 Oct 2008 19:41:39 +1000 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: Scott Long In-Reply-To: <48F5053D.7070705@samsco.org> Message-ID: <20081015184833.N43215@delplex.bde.org> References: <200810142028.m9EKShoL015514@svn.freebsd.org> <48F5053D.7070705@samsco.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: svn-src-head@FreeBSD.org, Marius Strobl , src-committers@FreeBSD.org, svn-src-all@FreeBSD.org Subject: Re: svn commit: r183896 - head/sys/dev/bge X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Oct 2008 19:00:27 -0000 On Tue, 14 Oct 2008, Scott Long wrote: > Marius Strobl wrote: >> Author: marius >> Date: Tue Oct 14 20:28:42 2008 >> New Revision: 183896 >> URL: http://svn.freebsd.org/changeset/base/183896 >> >> Log: >> Use bus_{read,write}_4(9) instead of bus_space_{read,write}_4(9) >> in order to get rid of the bus space handle and tag in the softc. > > Has anyone looked at the generated code from this interface switch and > compared it what was getting generated previously? Way back when, I just looked at the source code. This seems to be only a small pessimization since it involves just one extra indirection and related loss of optimization possibilities: (sc->sc_bt, sc->sc_bh) becomes r = sc->sc_br; (r->r_bustag, r->r_bushandle). In theory, the compiler could optimize by caching the tag and handle in registers in either case so that only 1 extra indirection is needed per function, but I've never seen that being done enough to make much difference. However, some drivers (e.g., ata) were too stupid to cache sc_bt and sc_bh, and converting these to use bus_nonspace*() was a relatively large optimization: r = sc->sc_br; (rman_get_bustag(r), rman_get_bushandle(r)) became r = sc->sc_br; (r->r_bustag, r->r_bushandle). Since rman_get_bustag() and rman_get_bushandle() have never been inline, calling both of them on every bus space access gave enormous code space and instruction count bloat and corresponding (relatively tiny) runtime bloat. The instructions normally run so much faster than the i/o that you can do hundreds or thousands of them per i/o before noticing the runtime bloat, and the instruction count bloat here is only about a factor of 20. > including made bus_space_read|write_4() compile > into a direct memory access on machines that supported it. The dubious > removal of bus_memio.h and bus_pio.h took away that benefit, and I'm > afraid that it's only getting worse now. Bus writes to card memory are > still very important to high-performance devices and shouldn't be > pessimized in the name of simpler-looking C code. I hate the bloat too, but it rarely matters (see above). I mainly noticed it using ddb. Where nice drivers are optimized to use a single instruction per i/o (or perhaps 2 with minimal bus space bloat), the ones that use rman_get*() on every i/o take 20+, and with a primitive debugger like ddb it is painful to even skip over all the function calls. Which devices have fast enough i/o for the extra indirection to matter? bge tries hard to avoid all PCI access in time-critical code. I think we reduced the PCI accesses to 1 PCI write per interrupt. Most device accesses for bge use host memory which is DMAed to/from by the hardware, so bus space doesn't apply. This type of access seems to be the best or only way for a PCI device to go fast enough (though it is still too slow on at least i386 since the DMA always causes cache misses). Bruce