From owner-freebsd-hardware@FreeBSD.ORG Wed Oct 1 11:57:57 2003 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9626A16A4B3; Wed, 1 Oct 2003 11:57:57 -0700 (PDT) Received: from mail.sandvine.com (sandvine.com [199.243.201.138]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4367343FF7; Wed, 1 Oct 2003 11:57:54 -0700 (PDT) (envelope-from rkim@sandvine.com) Received: by mail.sandvine.com with Internet Mail Service (5.5.2653.19) id ; Wed, 1 Oct 2003 14:57:53 -0400 Message-ID: From: Richy Kim To: "'freebsd-scsi@freebsd.org'" Date: Wed, 1 Oct 2003 14:57:52 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" cc: "'freebsd-hardware@freebsd.org'" Subject: RE: SCSI, SMP, and Supermicro, problems! X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Oct 2003 18:57:57 -0000 David, patch below basically compacts or unswizzles the pin-pin to 1:1 mapping when the mptable is parsed, thus saving a few interrupts... This is more so a workaround than a solution to FreeBSD's (4.x and older) historical interrupt limits. -r. Index: mp_machdep.c =================================================================== RCS file: /cvs/src/sys/i386/i386/mp_machdep.c,v retrieving revision 1.115.2.12 retrieving revision 1.115.2.12.1000.1 diff -u -r1.115.2.12 -r1.115.2.12.1000.1 --- mp_machdep.c 28 Apr 2002 18:18:17 -0000 1.115.2.12 +++ mp_machdep.c 14 Aug 2003 15:33:01 -0000 1.115.2.12.1000.1 @@ -27,6 +27,7 @@ #include "opt_cpu.h" #include "opt_user_ldt.h" +#include "opt_unswizzle.h" #ifdef SMP #include @@ -1404,6 +1405,30 @@ int_entry(int_entry_ptr entry, int intr) { int apic; +#ifdef UNSWIZZLE + int src_pin = entry->src_bus_irq & 3; + int dst_pin = entry->dst_apic_int & 3; + int slot = (entry->src_bus_irq >> 2) & 0x1f; + + if (src_pin != dst_pin) { + /* not 1:1 mapping, possibly a swizzled (pci) device behind a bridge */ + + /* swizzle src pin based pci 1.1 spec */ + int pin = (src_pin + slot) % 4; + + if (pin == dst_pin) { + /* BIOS has properly swizzled/rotated pins for this device + * However, we will un-swizzle the pin to get 1:1 mapping. + * This will save us some interrupts in cases where all + * slots behind the bridge use only first two pins anyways. + */ + + printf("IOAPIC: unswizzle intpin %d", entry->dst_apic_int); + entry->dst_apic_int = (entry->dst_apic_int & ~3) + src_pin; + printf(" --> %d\n", entry->dst_apic_int); + } + } +#endif io_apic_ints[intr].int_type = entry->int_type; io_apic_ints[intr].int_flags = entry->int_flags; -----Original Message----- From: Don Bowman Sent: Wednesday, October 01, 2003 2:42 PM To: 'David Raistrick'; Richy Kim; freebsd-scsi@freebsd.org Cc: freebsd-hardware@freebsd.org Subject: RE: SCSI, SMP, and Supermicro, problems! From: David Raistrick [mailto:drais@wow.atlasta.net] > > Since the above, I rebooted the system again...and for the > first time I > > have a running SMP system. > > ...which degraded into being hung right after I sent the last email: > > ahd0: Timedout SCB already complete. Interrupts may not be > functioning. > > ahd0: Timedout SCB already complete. Interrupts may not be > functioning. > Well, another suggestion has to do with the use of the APIC and interrupt rotation. I can't recall the exact details of this, we ended up adding a 'options UNSWIZZLE' to the kernel. This was specifically for stuff behind a bridge. The problem was that BSD only allowed 24 interrupts, and we had run out. --don