From owner-freebsd-current@FreeBSD.ORG Sun Apr 10 19:27:49 2005 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F068316A4CE for ; Sun, 10 Apr 2005 19:27:48 +0000 (GMT) Received: from mail22.sea5.speakeasy.net (mail22.sea5.speakeasy.net [69.17.117.24]) by mx1.FreeBSD.org (Postfix) with ESMTP id 97DD543D53 for ; Sun, 10 Apr 2005 19:27:48 +0000 (GMT) (envelope-from jhb@FreeBSD.org) Received: (qmail 24488 invoked from network); 10 Apr 2005 19:27:48 -0000 Received: from server.baldwin.cx ([216.27.160.63]) (envelope-sender )AES256-SHA encrypted SMTP for ; 10 Apr 2005 19:27:47 -0000 Received: from slimer.baldwin.cx (slimer.baldwin.cx [192.168.0.16]) (authenticated bits=0) by server.baldwin.cx (8.13.1/8.13.1) with ESMTP id j3AJRggs006229; Sun, 10 Apr 2005 15:27:42 -0400 (EDT) (envelope-from jhb@FreeBSD.org) From: John Baldwin To: Doug White Date: Sun, 10 Apr 2005 15:23:56 -0400 User-Agent: KMail/1.8 References: <20050406233405.O47071@carver.gumbysoft.com> <200504081656.51917.jhb@FreeBSD.org> <20050409231647.Y74901@carver.gumbysoft.com> In-Reply-To: <20050409231647.Y74901@carver.gumbysoft.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200504101523.56787.jhb@FreeBSD.org> X-Spam-Status: No, score=-2.8 required=4.2 tests=ALL_TRUSTED autolearn=failed version=3.0.2 X-Spam-Checker-Version: SpamAssassin 3.0.2 (2004-11-16) on server.baldwin.cx cc: freebsd-current@FreeBSD.org Subject: Re: Potential source of interrupt aliasing X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Apr 2005 19:27:49 -0000 On Sunday 10 April 2005 02:22 am, Doug White wrote: > On Fri, 8 Apr 2005, John Baldwin wrote: > > > A quick hack would be to blacklist intpin 16 on the first IOAPIC and > > > bump any PCI devices that ACPI says are claiming that interrupt. I > > > don't know how difficult this is to do with ACPI. How to handle this > > > for ATPIC mode is a little difficult since the boot interrupt either > > > gets routed to IRQ9 or ends up as a stray on IRQ7 (on my SCB2 at least > > > -- other boards may vary). > > > > You can't just move APIC interrupts around. They are physically wired > > that way. ATPIC mode won't matter. If there is a way to disable this in > > hardware, then it needs to be done in the _PIC method of the BIOS for > > ACPI. I don't think there's any way to allow for this for non-ACPI though > > as the MPTable spec only mentions the IMCR which these boards don't > > implement. > > I didn't think so, at least in the general case. It may be possible to > reprogram ICH registers to have usb use other INT lines if they're coming > up through SERIRQ and aren't physically wired to the pin, but I haven't > dug through the docs for that. > > For fun, I put together this one-liner to at least test the theory: > > http://people.freebsd.org/~dwhite/intr_machdep.c.20050409.patch Doing this should result in an instant interrupt storm since your PCI interrupts (which are level triggered) should now keep firing over and over and the machine will spend all it's time rescheduling the ithread. If it's not doing that then this is very curious. > This makes the aliased interrupts go away at the cost of hitting > sched_lock for each ithread interrupt. I'm trying to dream up a way of > telling if the ithread is active or scheduled so we don't need to bump it > again with setrunqueue(). I think I can use the it_need sparkplug to at > least save mutex grabs on interrupts that fire much faster than their > ithread handlers cycle. > > I'd like better granularity, though, but I'm not sure that setting a flag > that is cleared just before mi_switch() in ithread_loop() and set just > after won't race against an interrupt checking it to see if it needs to > setrunqueue() the ithread. Maybe clear the flag before trying to grab > sched_lock? it_need already does this. You haven't added any extra sched_locks. What've you done is change the interrupt code to never mask interrupt sources. If you aren't getting an interrupt storm on your first PCI interrupt then that is very curious indeed. Have you tried this with SMP disabled so that you only have one CPU running? It may be that due to SMP your ithread still gets a chance to run on one CPU while the other CPU is stormed with interrupt requests. Note that since storm detection is done in the ithread, this type of storm won't be detected and throttled. It also might not show up in the vmstats since we don't use atomic ops there anymore depending on whether or not the interrupt bounces between CPUs that keep overwriting the count on top of each other. > I'm also not sure how much this change affects system performance. I guess > I should hit up rwatson for some info on MUTEX_PROFILING. :) First see how the system acts with only one CPU. -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve" = http://www.FreeBSD.org