From owner-freebsd-hackers@FreeBSD.ORG Wed Sep 17 15:12:51 2003 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B19F616A4B3 for ; Wed, 17 Sep 2003 15:12:51 -0700 (PDT) Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 917D743F75 for ; Wed, 17 Sep 2003 15:12:50 -0700 (PDT) (envelope-from gallatin@cs.duke.edu) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.12.10/8.12.10) with ESMTP id h8HMCn5J018858 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO) for ; Wed, 17 Sep 2003 18:12:49 -0400 (EDT) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.11.6/8.9.1) id h8HMCit59578; Wed, 17 Sep 2003 18:12:44 -0400 (EDT) (envelope-from gallatin@cs.duke.edu) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <16232.56412.745069.248974@grasshopper.cs.duke.edu> Date: Wed, 17 Sep 2003 18:12:44 -0400 (EDT) To: freebsd-hackers@freebsd.org X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Subject: PCI interrupts passing DMA X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Sep 2003 22:12:51 -0000 I was toying with a programmable PCI card and wrote some code which DMAs a small block of data to the host, and then interrupts the host. The host checks the end of the block, and sees if it gets the value it expects. On an SMP P4 (hyperthreaded, with ServerWorks chipset) FreeBSD 4.8 UP, and on Linux 2.4.18, there is a huge delay between the interrupt being handled, and the DMA finally completing (from the host's perspective). Time enough for the interrupt handler to be triggered 3 or 4 times, and to print "foo" to a serial console line each time it notices that the DMA has not completed. The interesting thing is that on FreeBSD 4.8SMP, and FreeBSD 5.1-current (SMP), the data has arrived by the time the interrupt handler is called. This would be easy to explain if the interrupt latency were vastly different between the FreeBSD SMP kernel and the other kernels, but it does not seem to be. It actually seems to be about 5us faster (interrupt to wakeup of user-level process, so some fat is in there) than the FreeBSD UP kernel, possibly due to APIC io. *measurement done without console printf* My question is: What the heck could the SMP kernel be doing which causes the DMA to "complete" faster? Thanks, Drew