Date: Thu, 10 Apr 2008 17:31:53 -0500 From: Karl Denninger <karl@denninger.net> To: freebsd-stable@freebsd.org Subject: Disk I/O Question 7.0-STABLE Message-ID: <20080410223153.GA336@FS.denninger.net>
next in thread | raw e-mail | index | archive | help
FreeBSD 7.0-STABLE #9: Sat Mar 29 08:31:56 CST 2008 karl@FS.denninger.net:/usr/obj/usr/src/sys/KSD-SMP
Running generally well.
I have disks on the internal ICH7 adapters (on the motherboard), SATA, and
also a TWE controller with two disks.
When hitting the TWE controller hard I can hose the I/O performance on the
primary (onboard) adapter quite severely to the point that it seriously
impairs the system's overall performance. "systat -vm" shows the primary
channels going to 100% busy but the transaction and I/O rate count is not
all that high. The twe disk I am hammering, of course, is saturated (and
blowing data at a high rate of speed - 70MBps+)
It would seem to me that shouldn't be possible in that I can't possibly be
saturating the processor DMA capability with that sort of load - but it is.
Am I using a controller with a driver that is doing "not so good things"?
The machine in question is a quad-core - here's the boot info
Copyright (c) 1992-2008 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 7.0-STABLE #9: Sat Mar 29 08:31:56 CST 2008
karl@FS.denninger.net:/usr/obj/usr/src/sys/KSD-SMP
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Core(TM)2 Quad CPU @ 2.40GHz (2409.70-MHz 686-class CPU)
Origin = "GenuineIntel" Id = 0x6f7 Stepping = 7
Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Features2=0xe3bd<SSE3,RSVD2,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM>
AMD Features=0x20100000<NX,LM>
AMD Features2=0x1<LAHF>
Cores per package: 4
real memory = 2146082816 (2046 MB)
avail memory = 2089783296 (1992 MB)
ACPI APIC Table: <INTEL D975XBX2>
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
cpu0 (BSP): APIC ID: 0
cpu1 (AP): APIC ID: 1
cpu2 (AP): APIC ID: 2
cpu3 (AP): APIC ID: 3
ioapic0: Changing APIC ID to 2
ioapic0 <Version 2.0> irqs 0-23 on motherboard
kbd1 at kbdmux0
ath_hal: 0.9.20.3 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413)
acpi0: <INTEL D975XBX2> on motherboard
acpi0: [ITHREAD]
acpi0: Power Button (fixed)
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0
cpu0: <ACPI CPU> on acpi0
est0: <Enhanced SpeedStep Frequency Control> on cpu0
p4tcc0: <CPU Frequency Thermal Control> on cpu0
cpu1: <ACPI CPU> on acpi0
est1: <Enhanced SpeedStep Frequency Control> on cpu1
p4tcc1: <CPU Frequency Thermal Control> on cpu1
cpu2: <ACPI CPU> on acpi0
est2: <Enhanced SpeedStep Frequency Control> on cpu2
p4tcc2: <CPU Frequency Thermal Control> on cpu2
cpu3: <ACPI CPU> on acpi0
est3: <Enhanced SpeedStep Frequency Control> on cpu3
p4tcc3: <CPU Frequency Thermal Control> on cpu3
acpi_button0: <Sleep Button> on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <PCI-PCI bridge> at device 1.0 on pci0
pci1: <PCI bus> on pcib1
vgapci0: <VGA-compatible display> mem 0x91000000-0x91ffffff,0x80000000-0x8fffffff,0x90000000-0x90ffffff irq 16 at device 0.0 on pci1
pcib2: <PCI-PCI bridge> at device 3.0 on pci0
pci2: <PCI bus> on pcib2
pcib3: <PCI-PCI bridge> at device 0.0 on pci2
pci3: <PCI bus> on pcib3
puc0: <Oxford Semiconductor OX16PCI954 UARTs> port 0x4060-0x407f,0x4040-0x405f mem 0x92d03000-0x92d03fff,0x92d02000-0x92d02fff irq 16 at device 0.0 on pci3
puc0: [FILTER]
uart0: <16550 or compatible> on puc0
uart0: [FILTER]
uart1: <16550 or compatible> on puc0
uart1: [FILTER]
uart2: <16550 or compatible> on puc0
uart2: [FILTER]
uart3: <16550 or compatible> on puc0
uart3: [FILTER]
pci3: <bridge> at device 0.1 (no driver attached)
pcm0: <Intel 82801G High Definition Audio Controller> mem 0x92e00000-0x92e03fff irq 22 at device 27.0 on pci0
pcm0: [ITHREAD]
pcib4: <ACPI PCI-PCI bridge> at device 28.0 on pci0
pci4: <ACPI PCI bus> on pcib4
pcib5: <ACPI PCI-PCI bridge> at device 28.4 on pci0
pci5: <ACPI PCI bus> on pcib5
atapci0: <Marvell 88SX6145 UDMA133 controller> port 0x3018-0x301f,0x3024-0x3027,0x3010-0x3017,0x3020-0x3023,0x3000-0x300f mem 0x92c00000-0x92c003ff irq 16 at device 0.0 on pci5
atapci0: [ITHREAD]
ata2: <ATA channel 0> on atapci0
ata2: [ITHREAD]
ata3: <ATA channel 1> on atapci0
ata3: [ITHREAD]
pcib6: <ACPI PCI-PCI bridge> at device 28.5 on pci0
pci6: <ACPI PCI bus> on pcib6
em0: <Intel(R) PRO/1000 Network Connection Version - 6.7.3> port 0x2000-0x201f mem 0x92b00000-0x92b1ffff irq 17 at device 0.0 on pci6
em0: Using MSI interrupt
em0: Ethernet address: 00:19:d1:e3:b4:b8
em0: [FILTER]
uhci0: <UHCI (generic) USB controller> port 0x5080-0x509f irq 23 at device 29.0 on pci0
uhci0: [GIANT-LOCKED]
uhci0: [ITHREAD]
usb0: <UHCI (generic) USB controller> on uhci0
usb0: USB revision 1.0
uhub0: <Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb0
uhub0: 2 ports with 2 removable, self powered
uhci1: <UHCI (generic) USB controller> port 0x5060-0x507f irq 19 at device 29.1 on pci0
uhci1: [GIANT-LOCKED]
uhci1: [ITHREAD]
usb1: <UHCI (generic) USB controller> on uhci1
usb1: USB revision 1.0
uhub1: <Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb1
uhub1: 2 ports with 2 removable, self powered
uhci2: <UHCI (generic) USB controller> port 0x5040-0x505f irq 18 at device 29.2 on pci0
uhci2: [GIANT-LOCKED]
uhci2: [ITHREAD]
usb2: <UHCI (generic) USB controller> on uhci2
usb2: USB revision 1.0
uhub2: <Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb2
uhub2: 2 ports with 2 removable, self powered
uhci3: <UHCI (generic) USB controller> port 0x5020-0x503f irq 16 at device 29.3 on pci0
uhci3: [GIANT-LOCKED]
uhci3: [ITHREAD]
usb3: <UHCI (generic) USB controller> on uhci3
usb3: USB revision 1.0
uhub3: <Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb3
uhub3: 2 ports with 2 removable, self powered
ehci0: <Intel 82801GB/R (ICH7) USB 2.0 controller> mem 0x92e04400-0x92e047ff irq 23 at device 29.7 on pci0
ehci0: [GIANT-LOCKED]
ehci0: [ITHREAD]
usb4: EHCI version 1.0
usb4: companion controllers, 2 ports each: usb0 usb1 usb2 usb3
usb4: <Intel 82801GB/R (ICH7) USB 2.0 controller> on ehci0
usb4: USB revision 2.0
uhub4: <Intel EHCI root hub, class 9/0, rev 2.00/1.00, addr 1> on usb4
uhub4: 8 ports with 8 removable, self powered
pcib7: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci7: <ACPI PCI bus> on pcib7
fxp0: <Intel 82558 Pro/100 Ethernet> port 0x1000-0x101f mem 0x92a00000-0x92a00fff,0x92800000-0x928fffff irq 21 at device 0.0 on pci7
miibus0: <MII bus> on fxp0
inphy0: <i82555 10/100 media interface> PHY 1 on miibus0
inphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
fxp0: Ethernet address: 00:a0:c9:a4:78:c3
fxp0: [ITHREAD]
twe0: <3ware Storage Controller. Driver version 1.50.01.002> port 0x1020-0x102f mem 0x92904800-0x9290480f,0x92000000-0x927fffff irq 18 at device 2.0 on pci7
twe0: [GIANT-LOCKED]
twe0: [ITHREAD]
twe0: 2 ports, Firmware FE8S 1.05.00.068, BIOS BE7X 1.08.00.048
fwohci0: <Texas Instruments TSB43AB23> mem 0x92904000-0x929047ff,0x92900000-0x92903fff irq 18 at device 4.0 on pci7
fwohci0: [FILTER]
fwohci0: OHCI version 1.10 (ROM=0)
fwohci0: No. of Isochronous channels is 4.
fwohci0: EUI64 00:90:27:00:01:c8:f1:cc
fwohci0: Phy 1394a available S400, 3 ports.
fwohci0: Link S400, max_rec 2048 bytes.
firewire0: <IEEE1394(FireWire) bus> on fwohci0
dcons_crom0: <dcons configuration ROM> on firewire0
dcons_crom0: bus_addr 0x7c82c000
fwe0: <Ethernet over FireWire> on firewire0
if_fwe0: Fake Ethernet address: 02:90:27:c8:f1:cc
fwe0: Ethernet address: 02:90:27:c8:f1:cc
fwip0: <IP over FireWire> on firewire0
fwip0: Firewire address: 00:90:27:00:01:c8:f1:cc @ 0xfffe00000000, S400, maxrec 2048
sbp0: <SBP-2/SCSI over FireWire> on firewire0
fwohci0: Initiate bus reset
fwohci0: BUS reset
fwohci0: node_id=0xc800ffc0, gen=1, CYCLEMASTER mode
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
atapci1: <Intel ICH7 UDMA100 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x50b0-0x50bf irq 18 at device 31.1 on pci0
ata0: <ATA channel 0> on atapci1
ata0: [ITHREAD]
ata1: <ATA channel 1> on atapci1
ata1: [ITHREAD]
atapci2: <Intel ICH7 SATA300 controller> port 0x50c8-0x50cf,0x50e4-0x50e7,0x50c0-0x50c7,0x50e0-0x50e3,0x50a0-0x50af mem 0x92e04000-0x92e043ff irq 19 at device 31.2 on pci0
atapci2: [ITHREAD]
ata4: <ATA channel 0> on atapci2
ata4: [ITHREAD]
ata5: <ATA channel 1> on atapci2
ata5: [ITHREAD]
pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
atkbd0: [ITHREAD]
sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
sio0: type 16550A
sio0: [FILTER]
pmtimer0 on isa0
orm0: <ISA Option ROM> at iomem 0xcf000-0xcffff pnpid ORM0000 on isa0
ppc0: <Parallel port> at port 0x378-0x37f irq 7 on isa0
ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode
ppc0: FIFO with 16/16/8 bytes threshold
ppbus0: <Parallel port bus> on ppc0
ppbus0: [ITHREAD]
plip0: <PLIP network interface> on ppbus0
lpt0: <Printer> on ppbus0
lpt0: Interrupt-driven port
ppi0: <Parallel I/O> on ppbus0
ppc0: [GIANT-LOCKED]
ppc0: [ITHREAD]
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
sio1: configured irq 3 not in bitmap of probed irqs 0
sio1: port may not be enabled
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
ugen0: <American Power Conversion Smart-UPS 1500 FW:601.3.D USB FW:1.5, class 0/0, rev 1.10/0.06, addr 2> on uhub0
Timecounters tick every 1.000 msec
firewire0: 1 nodes, maxhop <= 0, cable IRM = 0 (me)
firewire0: bus manager 0 (me)
ipfw2 (+ipv6) initialized, divert enabled, nat loadable, rule-based forwarding disabled, default to deny, logging disabled
acd0: CDROM <GCR-8483B/1.00> at ata0-master UDMA33
ad8: 476940MB <Hitachi HDS725050KLA360 K2AOA51A> at ata4-master SATA150
ad10: 476940MB <Hitachi HDS725050KLA360 K2AOA51A> at ata5-master SATA150
pcm0: <HDA Codec: Sigmatel (Unknown)>
pcm0: <HDA Driver Revision: 20071129_0050>
twed0: <Unit 0, JBOD, Normal> on twe0
twed0: 953869MB (1953525168 sectors)
twed1: <Unit 1, JBOD, Normal> on twe0
twed1: 953869MB (1953525168 sectors)
GEOM_MIRROR: Force device b500 start due to timeout.
GEOM_MIRROR: Device mirror/b500 launched (2/3).
SMP: AP CPU #1 Launched!
SMP: AP CPU #2 Launched!
SMP: AP CPU #3 Launched!
Trying to mount root from ufs:/dev/mirror/b500a
em0: link state changed to UP
Ideas?
If the answer is "buy a different smart SATA controller of brand X" that's
fine so long as the name and model number follows :)
I can accomodate both PCI and PCI Express (if I remove the TWE card, which
is a PCI card)
Thanks in advance!
--
--
Karl Denninger (karl@denninger.net) Internet Consultant & Kids Rights Activist
http://www.denninger.net My home on the net - links to everything I do!
http://scubaforum.org Your UNCENSORED place to talk about DIVING!
http://genesis3.blogspot.com Musings Of A Sentient Mind
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20080410223153.GA336>