From owner-freebsd-net@FreeBSD.ORG Wed Jun 10 12:56:09 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8399B1065673 for ; Wed, 10 Jun 2009 12:56:09 +0000 (UTC) (envelope-from is@rambler-co.ru) Received: from mailrelay1.rambler.ru (mailrelay1.rambler.ru [81.19.66.239]) by mx1.freebsd.org (Postfix) with ESMTP id 14FBF8FC14 for ; Wed, 10 Jun 2009 12:56:08 +0000 (UTC) (envelope-from is@rambler-co.ru) Received: from kas30pipe.localhost (localhost [127.0.0.1]) by mailrelay1.rambler.ru (Postfix) with ESMTP id 926E8130C95; Wed, 10 Jun 2009 16:37:44 +0400 (MSD) Received: from localhost (is1.park.rambler.ru [81.19.64.121]) by mailrelay1.rambler.ru (Postfix) with ESMTP id 2739A130EF7; Wed, 10 Jun 2009 16:37:44 +0400 (MSD) Date: Wed, 10 Jun 2009 16:33:01 +0400 From: Igor Sysoev To: freebsd-net@freebsd.org Message-ID: <20090610123301.GE40250@rambler-co.ru> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="fdj2RfSjLxBAspz7" Content-Disposition: inline User-Agent: Mutt/1.5.13 (2006-08-11) X-Anti-Virus: Kaspersky Anti-Virus for MailServers 5.5.33/RELEASE, bases: 10062009 #2334115, status: clean X-SpamTest-Envelope-From: is@rambler-co.ru X-SpamTest-Group-ID: 00000000 X-SpamTest-Info: Profiles 8698 [Jun 09 2009] X-SpamTest-Info: {received from trusted relay: common white list} X-SpamTest-Method: white ip list X-SpamTest-Rate: 0 X-SpamTest-Status: Trusted X-SpamTest-Status-Extended: trusted X-SpamTest-Version: SMTP-Filter Version 3.0.0 [0284], KAS30/Release Cc: Subject: bge interrupt coalescing sysctls X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Jun 2009 12:56:09 -0000 --fdj2RfSjLxBAspz7 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline For a long time I used Bruce Evans' patch to tune bge interrupt coalescing: http://lists.freebsd.org/pipermail/freebsd-net/2007-November/015956.html However, recent commit SVN r192478 in 7-STABLE (r192127 in HEAD) had broken the patch. I'm not sure how to fix the collision, and since I do not use dynamic tuning I has left only static coalescing parameters in the patch and has added a loader tunable to set number of receive descriptors and read only sysctl to read the tunable. I usually use these parameters: /boot/loader.conf: hw.bge.rxd=512 /etc/sysctl.conf: dev.bge.0.rx_coal_ticks=500 dev.bge.0.tx_coal_ticks=10000 dev.bge.0.rx_max_coal_bds=64 dev.bge.0.tx_max_coal_bds=128 # apply the above parameters dev.bge.0.program_coal=1 Could anyone commit it ? -- Igor Sysoev http://sysoev.ru/en/ --fdj2RfSjLxBAspz7 Content-Type: text/plain; charset=koi8-r Content-Disposition: attachment; filename="patch.bge" --- sys/dev/bge/if_bge.c 2009-05-21 01:17:10.000000000 +0400 +++ sys/dev/bge/if_bge.c 2009-06-05 13:45:49.000000000 +0400 @@ -447,12 +447,16 @@ DRIVER_MODULE(miibus, bge, miibus_driver, miibus_devclass, 0, 0); static int bge_allow_asf = 0; +static int bge_rxd = BGE_SSLOTS; TUNABLE_INT("hw.bge.allow_asf", &bge_allow_asf); +TUNABLE_INT("hw.bge.rxd", &bge_rxd); SYSCTL_NODE(_hw, OID_AUTO, bge, CTLFLAG_RD, 0, "BGE driver parameters"); SYSCTL_INT(_hw_bge, OID_AUTO, allow_asf, CTLFLAG_RD, &bge_allow_asf, 0, "Allow ASF mode if available"); +SYSCTL_INT(_hw_bge, OID_AUTO, bge_rxd, CTLFLAG_RD, &bge_rxd, 0, + "Number of receive descriptors"); #define SPARC64_BLADE_1500_MODEL "SUNW,Sun-Blade-1500" #define SPARC64_BLADE_1500_PATH_BGE "/pci@1f,700000/network@2" @@ -1008,21 +1012,15 @@ return (0); } -/* - * The standard receive ring has 512 entries in it. At 2K per mbuf cluster, - * that's 1MB or memory, which is a lot. For now, we fill only the first - * 256 ring entries and hope that our CPU is fast enough to keep up with - * the NIC. - */ static int bge_init_rx_ring_std(struct bge_softc *sc) { int i; - for (i = 0; i < BGE_SSLOTS; i++) { + for (i = 0; i < bge_rxd; i++) { if (bge_newbuf_std(sc, i, NULL) == ENOBUFS) return (ENOBUFS); - }; + } bus_dmamap_sync(sc->bge_cdata.bge_rx_std_ring_tag, sc->bge_cdata.bge_rx_std_ring_map, @@ -2383,6 +2381,52 @@ #endif static int +bge_sysctl_program_coal(SYSCTL_HANDLER_ARGS) +{ + struct bge_softc *sc; + int error, i, val; + + val = 0; + error = sysctl_handle_int(oidp, &val, 0, req); + if (error != 0 || req->newptr == NULL) + return (error); + sc = arg1; + BGE_LOCK(sc); + + /* XXX cut from bge_blockinit(): */ + + /* Disable host coalescing until we get it set up */ + CSR_WRITE_4(sc, BGE_HCC_MODE, 0x00000000); + + /* Poll to make sure it's shut down. */ + for (i = 0; i < BGE_TIMEOUT; i++) { + if (!(CSR_READ_4(sc, BGE_HCC_MODE) & BGE_HCCMODE_ENABLE)) + break; + DELAY(10); + } + + if (i == BGE_TIMEOUT) { + device_printf(sc->bge_dev, + "host coalescing engine failed to idle\n"); + CSR_WRITE_4(sc, BGE_HCC_MODE, BGE_HCCMODE_ENABLE); + BGE_UNLOCK(sc); + return (ENXIO); + } + + /* Set up host coalescing defaults */ + CSR_WRITE_4(sc, BGE_HCC_RX_COAL_TICKS, sc->bge_rx_coal_ticks); + CSR_WRITE_4(sc, BGE_HCC_TX_COAL_TICKS, sc->bge_tx_coal_ticks); + CSR_WRITE_4(sc, BGE_HCC_RX_MAX_COAL_BDS, sc->bge_rx_max_coal_bds); + CSR_WRITE_4(sc, BGE_HCC_TX_MAX_COAL_BDS, sc->bge_tx_max_coal_bds); + + /* Turn on host coalescing state machine */ + CSR_WRITE_4(sc, BGE_HCC_MODE, BGE_HCCMODE_ENABLE); + + BGE_UNLOCK(sc); + return (0); +} + +static int bge_attach(device_t dev) { struct ifnet *ifp; @@ -4495,6 +4539,19 @@ ctx = device_get_sysctl_ctx(sc->bge_dev); children = SYSCTL_CHILDREN(device_get_sysctl_tree(sc->bge_dev)); + SYSCTL_ADD_PROC(ctx, children, OID_AUTO, "program_coal", + CTLTYPE_INT | CTLFLAG_RW, + sc, 0, bge_sysctl_program_coal, "I", + "program bge coalescence values"); + SYSCTL_ADD_UINT(ctx, children, OID_AUTO, "rx_coal_ticks", CTLFLAG_RW, + &sc->bge_rx_coal_ticks, 0, ""); + SYSCTL_ADD_UINT(ctx, children, OID_AUTO, "tx_coal_ticks", CTLFLAG_RW, + &sc->bge_tx_coal_ticks, 0, ""); + SYSCTL_ADD_UINT(ctx, children, OID_AUTO, "rx_max_coal_bds", CTLFLAG_RW, + &sc->bge_rx_max_coal_bds, 0, ""); + SYSCTL_ADD_UINT(ctx, children, OID_AUTO, "tx_max_coal_bds", CTLFLAG_RW, + &sc->bge_tx_max_coal_bds, 0, ""); + #ifdef BGE_REGISTER_DEBUG SYSCTL_ADD_PROC(ctx, children, OID_AUTO, "debug_info", CTLTYPE_INT | CTLFLAG_RW, sc, 0, bge_sysctl_debug_info, "I", --fdj2RfSjLxBAspz7--