Date: Thu, 18 Sep 2003 10:20:38 -0400 (EDT) From: Andrew Gallatin <gallatin@cs.duke.edu> To: aaro@iki.fi Cc: freebsd-hackers@freebsd.org Subject: Re: PCI interrupts passing DMA Message-ID: <16233.48950.348205.388570@grasshopper.cs.duke.edu> In-Reply-To: <Pine.OSF.4.58.0309181412540.10256@sirppi.helsinki.fi> References: <Pine.OSF.4.58.0309181412540.10256@sirppi.helsinki.fi>
next in thread | previous in thread | raw e-mail | index | archive | help
Aaro Koskinen writes:
> > My question is: What the heck could the SMP kernel be doing which
> > causes the DMA to "complete" faster?
>
> The chipset probably uses PCI bus (MSI-like mechanism) to deliver the
> interrupt from the IO APIC to the local APIC, which means that the PCI
> bridge(s) must complete the DMA transfer before the interrupt is
> delivered to preserve the write order.
AHA! I think you hit it on the nose. It turns out that the FreeBSD
SMP kernel sets up all IOAPIC interrupts as IOART_DELLOPRI. But
linux doesn't set the IOART_DELLOPRI bit. This seems account for the
difference in behaviour between FreeBSD & linux.
The following diff seems to make SMP FreeBSD behave the same as linux,
and the same as UP FreeBSD:
Index: i386/i386/mpapic.c
===================================================================
RCS file: /home/ncvs/src/sys/i386/i386/mpapic.c,v
retrieving revision 1.63
diff -u -r1.63 mpapic.c
--- i386/i386/mpapic.c 23 Jul 2003 18:59:38 -0000 1.63
+++ i386/i386/mpapic.c 18 Sep 2003 14:07:38 -0000
@@ -134,7 +134,7 @@
((u_int32_t) \
(IOART_INTMSET | \
IOART_DESTPHY | \
- IOART_DELLOPRI))
+ IOART_DELFIXED))
#define DEFAULT_ISA_FLAGS \
((u_int32_t) \
> In PIC mode, the interrupt is delivered by the wire and it has no
> effect on pending writes. A common solution is that the interrupt
> handler must perform a read from the device to the force flushing of
> buffers.
Yep. I was trying to avoid that because PIO reads are so horribly
expensive.. I guess I'll have to do it after all. I wish MSIs had
been around from the beginning & were more widely used.
Thanks for your help,
Drew
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?16233.48950.348205.388570>