From owner-freebsd-bugs  Tue Nov 27 11:30:15 2001
Delivered-To: freebsd-bugs@hub.freebsd.org
Received: from freefall.freebsd.org (freefall.FreeBSD.org [216.136.204.21])
	by hub.freebsd.org (Postfix) with ESMTP id 36E0937B416
	for <freebsd-bugs@hub.freebsd.org>; Tue, 27 Nov 2001 11:30:01 -0800 (PST)
Received: (from gnats@localhost)
	by freefall.freebsd.org (8.11.6/8.11.6) id fARJU1b85855;
	Tue, 27 Nov 2001 11:30:01 -0800 (PST)
	(envelope-from gnats)
Received: from freefall.freebsd.org (freefall.FreeBSD.org [216.136.204.21])
	by hub.freebsd.org (Postfix) with ESMTP id C323637B419
	for <freebsd-gnats-submit@FreeBSD.org>; Tue, 27 Nov 2001 11:28:25 -0800 (PST)
Received: (from nobody@localhost)
	by freefall.freebsd.org (8.11.6/8.11.6) id fARJSP584881;
	Tue, 27 Nov 2001 11:28:25 -0800 (PST)
	(envelope-from nobody)
Message-Id: <200111271928.fARJSP584881@freefall.freebsd.org>
Date: Tue, 27 Nov 2001 11:28:25 -0800 (PST)
From: David S Madole <david@madole.net>
To: freebsd-gnats-submit@FreeBSD.org
X-Send-Pr-Version: www-1.0
Subject: kern/32338: Network to disk write performance low under ATA with DMA
Sender: owner-freebsd-bugs@FreeBSD.ORG
Precedence: bulk
List-ID: <freebsd-bugs.FreeBSD.ORG>
List-Archive: <http://docs.freebsd.org/mail/> (Web Archive)
List-Help: <mailto:majordomo@FreeBSD.ORG?subject=help> (List Instructions)
List-Subscribe: <mailto:majordomo@FreeBSD.ORG?subject=subscribe%20freebsd-bugs>
List-Unsubscribe: <mailto:majordomo@FreeBSD.ORG?subject=unsubscribe%20freebsd-bugs>
X-Loop: FreeBSD.org


>Number:         32338
>Category:       kern
>Synopsis:       Network to disk write performance low under ATA with DMA
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue Nov 27 11:30:01 PST 2001
>Closed-Date:
>Last-Modified:
>Originator:     David S Madole
>Release:        4.4-RELEASE
>Organization:
>Environment:
FreeBSD 4.4-RELEASE #4: Mon Nov 26 18:39:35 GMT 2001
    dmadole@devel.madole.net:/usr/src/sys/compile/MADOLE
Timecounter "i8254"  frequency 1193182 Hz
Timecounter "TSC"  frequency 551252600 Hz
CPU: AMD-K6(tm)-III Processor (551.25-MHz 586-class CPU)
  Origin = "AuthenticAMD"  Id = 0x5d0  Stepping = 0
  Features=0x8021bf<FPU,VME,DE,PSE,TSC,MSR,MCE,CX8,PGE,MMX>
  AMD Features=0xc0000800<SYSCALL,DSP,3DNow!>
real memory  = 134217728 (131072K bytes)
avail memory = 127799296 (124804K bytes)
Preloaded elf kernel "kernel" at 0xc02d1000.
K6-family MTRR support enabled (2 registers)
Using $PIR table, 6 entries at 0xc00fddf0
npx0: <math processor> on motherboard
npx0: INT 16 interface
pcib0: <Host to PCI bridge> on motherboard
pci0: <PCI bus> on pcib0
pcib2: <VIA 82C598MVP (Apollo MVP3) PCI-PCI (AGP) bridge> at device 1.0 on pci0
pci1: <PCI bus> on pcib2
isab0: <VIA 82C586 PCI-ISA bridge> at device 7.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <VIA 82C586 ATA33 controller> port 0xd000-0xd00f at device 7.1 on pci0
ata0: at 0x1f0 irq 14 on atapci0
ata1: at 0x170 irq 15 on atapci0
pci0: <Cirrus Logic GD5434 SVGA controller> at 8.0
ed0: <NE2000 PCI Ethernet (RealTek 8029)> port 0xd800-0xd81f irq 11 at device 9.
0 on pci0
ed0: address 00:00:b4:9d:c5:08, type NE2000 (16 bit)
sis0: <NatSemi DP83815 10/100BaseTX> port 0xdc00-0xdcff mem 0xdf000000-0xdf000ff
f irq 9 at device 10.0 on pci0
sis0: Ethernet address: 00:a0:cc:75:1b:be
miibus0: <MII bus> on sis0
ukphy0: <Generic IEEE 802.3u media interface> on miibus0
ukphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
oltr0: <Olicom PCI/II 16/4 Adapter (OC-3137)> port 0xe000-0xe03f irq 10 at devic
e 11.0 on pci0
oltr0: MAC address 00:00:83:23:9c:e9
pcib1: <Host to PCI bridge> on motherboard
pci2: <PCI bus> on pcib1
orm0: <Option ROM> at iomem 0xc0000-0xc7fff on isa0
fdc0: <NEC 72065B or clone> at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on isa0
fdc0: FIFO enabled, 8 bytes threshold
fd0: <1440-KB 3.5" drive> on fdc0 drive 0
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0
sio0: type 16550A
sio1 at port 0x2f8-0x2ff irq 3 on isa0
sio1: type 16550A
sio2: configured irq 5 not in bitmap of probed irqs 0
ppc0: <Parallel port> at port 0x378-0x37f irq 7 drq 3 on isa0
ppc0: Generic chipset (ECP/PS2/NIBBLE) in COMPATIBLE mode
ppc0: FIFO with 16/16/16 bytes threshold
lpt0: <Printer> on ppbus0
lpt0: Interrupt-driven port
IP packet filtering initialized, divert enabled, rule-based forwarding enabled,
default to deny, logging disabled
ad0: 58623MB <Maxtor 96147U8> [119108/16/63] at ata0-master UDMA33
acd0: CD-RW <8X4X32> at ata1-master using PIO4
Mounting root from ufs:/dev/ad0s1a

>Description:
Data transfers from network to ATA hard drive are very slow. Visible
with Samba, HTTP PUT, etc., but most easily demonstrated with FTP.
Only occurs when DMA is enabled on the ATA controller.

Interestingly, only seems to happen when network data is being written
to the drive. Doing a large write to the drive while simultaneously
doing a 'ping -f -s 1400' to another node from another session does
not slow the disk writes significantly.

The drive is a Maxtor 60GB (model number in included dmesg). NIC is
NetGear with sis driver, although same problem occurs with LinkSys
card on dc driver. Also occurs with older Maxtor 6GB drive.

>How-To-Repeat:
Example below is FTPing to another machine also with 4.4-RELEASE
connected through NetGear NICs (sis driver) and a 100MB switch,
full-duplex.

Note that gets to the hard drive are extremely slow, even though
gets to /dev/null are fine. Puts are fine. The disk write speed
is OK when it is not data through the network. Turn off DMA and
write speed is fine (although CPU usage is very high!)

I am showing a small test file here, but transfer rates are 
similar with large (200MB) test files.

# sysctl hw.atamodes
hw.atamodes: dma,---,pio,---,
# ftp test.madole.net
Connected to test.madole.net.
220 test.madole.net FTP server (Version 6.00LS) ready.
Name (test.madole.net:dmadole): dmadole
331 Password required for dmadole.
Password:
230 User dmadole logged in.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> get /bin/csh /dev/null
local: /dev/null remote: /bin/csh
150 Opening BINARY mode data connection for '/bin/csh' (636076 bytes).
226 Transfer complete.
636076 bytes received in 0.13 seconds (4.52 MB/s)
ftp> get /bin/csh /tmp/temp
local: /tmp/temp remote: /bin/csh
150 Opening BINARY mode data connection for '/bin/csh' (636076 bytes).
100% |**************************************************|   621 KB    00:00 ETA
226 Transfer complete.
636076 bytes received in 16.04 seconds (38.73 KB/s)
ftp> put /bin/csh /tmp/temp
local: /bin/csh remote: /tmp/temp
150 Opening BINARY mode data connection for '/tmp/temp'.
100% |**************************************************|   621 KB    00:00 ETA
226 Transfer complete.
636076 bytes sent in 0.33 seconds (1.85 MB/s)
ftp> quit
221 Goodbye.
# dd if=/bin/csh of=/tmp/temp
1242+1 records in
1242+1 records out
636076 bytes transferred in 0.082739 secs (7687741 bytes/sec)
# sysctl -w hw.atamodes=pio,---,pio,---,
hw.atamodes: dma,---,pio,---, -> pio,---,pio,---,
# ftp test.madole.net
Connected to test.madole.net.
220 test.madole.net FTP server (Version 6.00LS) ready.
Name (test.madole.net:dmadole): dmadole
331 Password required for dmadole.
Password:
230 User dmadole logged in.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> get /bin/csh /tmp/temp
local: /tmp/temp remote: /bin/csh
150 Opening BINARY mode data connection for '/bin/csh' (636076 bytes).
100% |**************************************************|   621 KB    00:00 ETA
226 Transfer complete.
636076 bytes received in 0.12 seconds (5.12 MB/s)
ftp> quit
221 Goodbye.
#
>Fix:
I wish I knew where to look!

PIO mode is a kinda-workaround, but it hits the CPU hard enough to
cause other less-serious performance problems.

>Release-Note:
>Audit-Trail:
>Unformatted:

To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-bugs" in the body of the message