From owner-freebsd-bugs Fri Feb 8 18: 0:19 2002 Delivered-To: freebsd-bugs@hub.freebsd.org Received: from freefall.freebsd.org (freefall.FreeBSD.org [216.136.204.21]) by hub.freebsd.org (Postfix) with ESMTP id 6977037B404 for ; Fri, 8 Feb 2002 18:00:03 -0800 (PST) Received: (from gnats@localhost) by freefall.freebsd.org (8.11.6/8.11.6) id g19203V15631; Fri, 8 Feb 2002 18:00:03 -0800 (PST) (envelope-from gnats) Date: Fri, 8 Feb 2002 18:00:03 -0800 (PST) Message-Id: <200202090200.g19203V15631@freefall.freebsd.org> To: freebsd-bugs@FreeBSD.org Cc: From: Luigi Rizzo Subject: Re: kern/32338: Network to disk write performance low under ATA with DMA Reply-To: Luigi Rizzo Sender: owner-freebsd-bugs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org The following reply was made to PR kern/32338; it has been noted by GNATS. From: Luigi Rizzo To: "David S. Madole" Cc: sos@freebsd.dk, wpaul@FreeBSD.org, freebsd-gnats-submit@FreeBSD.org Subject: Re: kern/32338: Network to disk write performance low under ATA with DMA Date: Fri, 8 Feb 2002 17:57:48 -0800 Hi, sorry for the delay with which I reply but i only had a chance to test this now. It seems that the removal of sis_init from sis_rxeoc() is definitely an important part of the fix, as in presence of bidirectional traffic an overflow on the receive side will cause a reset of the chip and drop all packets queued on the transmit side. Bill, do you think the removal of sis_init() has ill side effects otherwise ? I am bombing the interface with 148kpps (on a 486-133 so there are plenty of overflows) and things seem to work fine, much better indeed than when sis_init() was included in sis_rxeoc(). Not sure about the PCI latency register, i would like to test this a bit more, as there are many reasons why rxeoc is called. cheers luigi On Mon, Jan 07, 2002 at 08:13:19PM -0500, David S. Madole wrote: > >===== Original Message From sos@freebsd.dk ===== > >It seems David S Madole wrote: > >> >Description: > >> Data transfers from network to ATA hard drive are very slow. Visible > >> with Samba, HTTP PUT, etc., but most easily demonstrated with FTP. > >> Only occurs when DMA is enabled on the ATA controller. > >> > >> Interestingly, only seems to happen when network data is being written > >> to the drive. Doing a large write to the drive while simultaneously > >> doing a 'ping -f -s 1400' to another node from another session does > >> not slow the disk writes significantly. > >> > >> The drive is a Maxtor 60GB (model number in included dmesg). NIC is > >> NetGear with sis driver, although same problem occurs with LinkSys > >> card on dc driver. Also occurs with older Maxtor 6GB drive. > > > >Hmm, this sounds like the SiS network card may be using DMA too > >and badly at that, making the net card and the ATA driver > >compete for the bus... > > ( my apologies if anyone has received this already, but I don't think the > copy I sent previously went out correctly since it never showed up on > gnats > and I was having mail trouble for a few days ) > > Bill, I copied you since this is your driver. > > Luigi, I apologize if inappropriate to copy you on this, but I see you've > been > active in the sis driver lately and thought you might be interested. > > Søren, thanks for the response. As I mentioned before, I did have this > problem with the dc driver as well, on an ADMtek AN985. > > It turns out that the NIC was unable to DMA it's FIFO into memory fast > enough, > causing it to overflow and packets to get lost, throttling back TCP. It > seems > that my BIOS initializes the PCI latency timer on the card to 32, which > doesn't let it move enough data at a time to keep up with 100Mbs while > competing with the ATA controller. > > I am heistant to second-quess every BIOS, so I decided to try an adaptive > approach and bump the timer up a little bit (32) each time an overflow > occurs. > On my machine, once it reaches 96 the card is stable and never overflows > again. This increased my FTP-to-disk throughput to a little over 9MBps, > pretty > good. I also changed the driver to not reinitialize the card when an > overflow > or receive error occurs -- it seems unnecessary and takes a relatively long > time, causing many packets to get lost. > > I also adjusted some NIC settings particular to the National chip to make > them > conditional on the silicon version, as the latest datasheet specifies. I'm > guessing it doesn't really matter much since these are probably just > defaults > on the later chips. > > I think some other DMA-based NIC drivers, like dc could probably benefit > from > this as well. I can fix up the dc driver as well, if this seems like the > best > way to go, and see if it fixes the issue there, too. > > By the way, the attached diff is against 4.4-RELEASE. > > Thanks, > Dave > > > *** if_sisreg.h.orig Wed Feb 21 22:17:51 2001 > --- if_sisreg.h Thu Nov 29 19:28:35 2001 > *************** > *** 76,81 **** > --- 76,82 ---- > > /* NS DP83815 registers */ > #define NS_CLKRUN 0x3C > + #define NS_SRR 0x58 > #define NS_BMCR 0x80 > #define NS_BMSR 0x84 > #define NS_PHYIDR1 0x88 > *************** > *** 401,406 **** > --- 402,408 ---- > struct sis_list_data *sis_ldata; > struct sis_ring_data sis_cdata; > struct callout_handle sis_stat_ch; > + device_t sis_dev; > }; > > /* > *** if_sis.c.orig Wed Feb 21 22:17:51 2001 > --- if_sis.c Thu Nov 29 19:41:31 2001 > *************** > *** 723,728 **** > --- 723,730 ---- > unit = device_get_unit(dev); > bzero(sc, sizeof(struct sis_softc)); > > + sc->sis_dev = dev; > + > if (pci_get_device(dev) == SIS_DEVICEID_900) > sc->sis_type = SIS_TYPE_900; > if (pci_get_device(dev) == SIS_DEVICEID_7016) > *************** > *** 1159,1166 **** > void sis_rxeoc(sc) > struct sis_softc *sc; > { > sis_rxeof(sc); > - sis_init(sc); > return; > } > > --- 1161,1183 ---- > void sis_rxeoc(sc) > struct sis_softc *sc; > { > + int latency; > + > + /* > + * The BIOS may have initialized the maximum latency timer > + * too low to be able to keep up with a 100Mbs stream when > + * heavy disk or other DMA is taking place. Try to correct > + * for this adaptively by bumping it up a little each time > + * the receive FIFO overflows. > + */ > + > + latency = pci_read_config(sc->sis_dev, PCIR_LATTIMER, 1); > + if (latency < 255) { > + if ((latency += 32) > 255) latency = 255; > + pci_write_config(sc->sis_dev, PCIR_LATTIMER, latency, 1); > + } > + > sis_rxeof(sc); > return; > } > > *************** > *** 1293,1305 **** > sis_txeof(sc); > > if ((status & SIS_ISR_RX_DESC_OK) || > ! (status & SIS_ISR_RX_OK)) > sis_rxeof(sc); > > ! if ((status & SIS_ISR_RX_ERR) || > ! (status & SIS_ISR_RX_OFLOW)) { > sis_rxeoc(sc); > - } > > if (status & SIS_ISR_SYSERR) { > sis_reset(sc); > --- 1310,1321 ---- > sis_txeof(sc); > > if ((status & SIS_ISR_RX_DESC_OK) || > ! (status & SIS_ISR_RX_OK) || > ! (status & SIS_ISR_RX_ERR)) > sis_rxeof(sc); > > ! if (status & SIS_ISR_RX_OFLOW) > sis_rxeoc(sc); > > if (status & SIS_ISR_SYSERR) { > sis_reset(sc); > *************** > *** 1562,1569 **** > * performance." Note however that at least three > * of the registers are listed as "reserved" in > * the register map, so who knows what they do. > */ > ! if (sc->sis_type == SIS_TYPE_83815) { > CSR_WRITE_4(sc, NS_PHY_PAGE, 0x0001); > CSR_WRITE_4(sc, NS_PHY_CR, 0x189C); > CSR_WRITE_4(sc, NS_PHY_TDATA, 0x0000); > --- 1578,1593 ---- > * performance." Note however that at least three > * of the registers are listed as "reserved" in > * the register map, so who knows what they do. > + * > + * An appararently later version (December 2000) > + * has this data on page 78 and qualifies it as > + * applying only to silicon version 203h, which > + * is aparently a typo'd reference to version 302h > + * as referred to on page 64. > */ > ! if (sc->sis_type == SIS_TYPE_83815 && > ! CSR_READ_4(sc, NS_SRR) == 0x0302) { > ! > CSR_WRITE_4(sc, NS_PHY_PAGE, 0x0001); > CSR_WRITE_4(sc, NS_PHY_CR, 0x189C); > CSR_WRITE_4(sc, NS_PHY_TDATA, 0x0000); To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-bugs" in the body of the message