Date: Fri, 8 Feb 2002 18:00:03 -0800 (PST) From: Luigi Rizzo <luigi@FreeBSD.org> To: freebsd-bugs@FreeBSD.org Subject: Re: kern/32338: Network to disk write performance low under ATA with DMA Message-ID: <200202090200.g19203V15631@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
The following reply was made to PR kern/32338; it has been noted by GNATS.
From: Luigi Rizzo <luigi@FreeBSD.org>
To: "David S. Madole" <david@madole.net>
Cc: sos@freebsd.dk, wpaul@FreeBSD.org,
freebsd-gnats-submit@FreeBSD.org
Subject: Re: kern/32338: Network to disk write performance low under ATA with DMA
Date: Fri, 8 Feb 2002 17:57:48 -0800
Hi,
sorry for the delay with which I reply but i only had a chance
to test this now.
It seems that the removal of sis_init from sis_rxeoc() is
definitely an important part of the fix, as in presence of
bidirectional traffic an overflow on the receive side will
cause a reset of the chip and drop all packets queued on
the transmit side.
Bill, do you think the removal of sis_init() has ill side
effects otherwise ? I am bombing the interface with 148kpps
(on a 486-133 so there are plenty of overflows)
and things seem to work fine, much better indeed than
when sis_init() was included in sis_rxeoc().
Not sure about the PCI latency register, i would like
to test this a bit more, as there are many reasons why
rxeoc is called.
cheers
luigi
On Mon, Jan 07, 2002 at 08:13:19PM -0500, David S. Madole wrote:
> >===== Original Message From sos@freebsd.dk =====
> >It seems David S Madole wrote:
> >> >Description:
> >> Data transfers from network to ATA hard drive are very slow. Visible
> >> with Samba, HTTP PUT, etc., but most easily demonstrated with FTP.
> >> Only occurs when DMA is enabled on the ATA controller.
> >>
> >> Interestingly, only seems to happen when network data is being written
> >> to the drive. Doing a large write to the drive while simultaneously
> >> doing a 'ping -f -s 1400' to another node from another session does
> >> not slow the disk writes significantly.
> >>
> >> The drive is a Maxtor 60GB (model number in included dmesg). NIC is
> >> NetGear with sis driver, although same problem occurs with LinkSys
> >> card on dc driver. Also occurs with older Maxtor 6GB drive.
> >
> >Hmm, this sounds like the SiS network card may be using DMA too
> >and badly at that, making the net card and the ATA driver
> >compete for the bus...
>
> ( my apologies if anyone has received this already, but I don't think the
> copy I sent previously went out correctly since it never showed up on
> gnats
> and I was having mail trouble for a few days )
>
> Bill, I copied you since this is your driver.
>
> Luigi, I apologize if inappropriate to copy you on this, but I see you've
> been
> active in the sis driver lately and thought you might be interested.
>
> Søren, thanks for the response. As I mentioned before, I did have this
> problem with the dc driver as well, on an ADMtek AN985.
>
> It turns out that the NIC was unable to DMA it's FIFO into memory fast
> enough,
> causing it to overflow and packets to get lost, throttling back TCP. It
> seems
> that my BIOS initializes the PCI latency timer on the card to 32, which
> doesn't let it move enough data at a time to keep up with 100Mbs while
> competing with the ATA controller.
>
> I am heistant to second-quess every BIOS, so I decided to try an adaptive
> approach and bump the timer up a little bit (32) each time an overflow
> occurs.
> On my machine, once it reaches 96 the card is stable and never overflows
> again. This increased my FTP-to-disk throughput to a little over 9MBps,
> pretty
> good. I also changed the driver to not reinitialize the card when an
> overflow
> or receive error occurs -- it seems unnecessary and takes a relatively long
> time, causing many packets to get lost.
>
> I also adjusted some NIC settings particular to the National chip to make
> them
> conditional on the silicon version, as the latest datasheet specifies. I'm
> guessing it doesn't really matter much since these are probably just
> defaults
> on the later chips.
>
> I think some other DMA-based NIC drivers, like dc could probably benefit
> from
> this as well. I can fix up the dc driver as well, if this seems like the
> best
> way to go, and see if it fixes the issue there, too.
>
> By the way, the attached diff is against 4.4-RELEASE.
>
> Thanks,
> Dave
>
>
> *** if_sisreg.h.orig Wed Feb 21 22:17:51 2001
> --- if_sisreg.h Thu Nov 29 19:28:35 2001
> ***************
> *** 76,81 ****
> --- 76,82 ----
>
> /* NS DP83815 registers */
> #define NS_CLKRUN 0x3C
> + #define NS_SRR 0x58
> #define NS_BMCR 0x80
> #define NS_BMSR 0x84
> #define NS_PHYIDR1 0x88
> ***************
> *** 401,406 ****
> --- 402,408 ----
> struct sis_list_data *sis_ldata;
> struct sis_ring_data sis_cdata;
> struct callout_handle sis_stat_ch;
> + device_t sis_dev;
> };
>
> /*
> *** if_sis.c.orig Wed Feb 21 22:17:51 2001
> --- if_sis.c Thu Nov 29 19:41:31 2001
> ***************
> *** 723,728 ****
> --- 723,730 ----
> unit = device_get_unit(dev);
> bzero(sc, sizeof(struct sis_softc));
>
> + sc->sis_dev = dev;
> +
> if (pci_get_device(dev) == SIS_DEVICEID_900)
> sc->sis_type = SIS_TYPE_900;
> if (pci_get_device(dev) == SIS_DEVICEID_7016)
> ***************
> *** 1159,1166 ****
> void sis_rxeoc(sc)
> struct sis_softc *sc;
> {
> sis_rxeof(sc);
> - sis_init(sc);
> return;
> }
>
> --- 1161,1183 ----
> void sis_rxeoc(sc)
> struct sis_softc *sc;
> {
> + int latency;
> +
> + /*
> + * The BIOS may have initialized the maximum latency timer
> + * too low to be able to keep up with a 100Mbs stream when
> + * heavy disk or other DMA is taking place. Try to correct
> + * for this adaptively by bumping it up a little each time
> + * the receive FIFO overflows.
> + */
> +
> + latency = pci_read_config(sc->sis_dev, PCIR_LATTIMER, 1);
> + if (latency < 255) {
> + if ((latency += 32) > 255) latency = 255;
> + pci_write_config(sc->sis_dev, PCIR_LATTIMER, latency, 1);
> + }
> +
> sis_rxeof(sc);
> return;
> }
>
> ***************
> *** 1293,1305 ****
> sis_txeof(sc);
>
> if ((status & SIS_ISR_RX_DESC_OK) ||
> ! (status & SIS_ISR_RX_OK))
> sis_rxeof(sc);
>
> ! if ((status & SIS_ISR_RX_ERR) ||
> ! (status & SIS_ISR_RX_OFLOW)) {
> sis_rxeoc(sc);
> - }
>
> if (status & SIS_ISR_SYSERR) {
> sis_reset(sc);
> --- 1310,1321 ----
> sis_txeof(sc);
>
> if ((status & SIS_ISR_RX_DESC_OK) ||
> ! (status & SIS_ISR_RX_OK) ||
> ! (status & SIS_ISR_RX_ERR))
> sis_rxeof(sc);
>
> ! if (status & SIS_ISR_RX_OFLOW)
> sis_rxeoc(sc);
>
> if (status & SIS_ISR_SYSERR) {
> sis_reset(sc);
> ***************
> *** 1562,1569 ****
> * performance." Note however that at least three
> * of the registers are listed as "reserved" in
> * the register map, so who knows what they do.
> */
> ! if (sc->sis_type == SIS_TYPE_83815) {
> CSR_WRITE_4(sc, NS_PHY_PAGE, 0x0001);
> CSR_WRITE_4(sc, NS_PHY_CR, 0x189C);
> CSR_WRITE_4(sc, NS_PHY_TDATA, 0x0000);
> --- 1578,1593 ----
> * performance." Note however that at least three
> * of the registers are listed as "reserved" in
> * the register map, so who knows what they do.
> + *
> + * An appararently later version (December 2000)
> + * has this data on page 78 and qualifies it as
> + * applying only to silicon version 203h, which
> + * is aparently a typo'd reference to version 302h
> + * as referred to on page 64.
> */
> ! if (sc->sis_type == SIS_TYPE_83815 &&
> ! CSR_READ_4(sc, NS_SRR) == 0x0302) {
> !
> CSR_WRITE_4(sc, NS_PHY_PAGE, 0x0001);
> CSR_WRITE_4(sc, NS_PHY_CR, 0x189C);
> CSR_WRITE_4(sc, NS_PHY_TDATA, 0x0000);
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-bugs" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200202090200.g19203V15631>