Date: Mon, 4 Feb 2008 09:25:37 -0800 From: Tech Lab Manager <tech@liveoaksf.org> To: freebsd-acpi@freebsd.org Subject: Re: SMP, ACPI and interrupt storm Message-ID: <1756A487-5F0B-466D-921D-B9F6B6FF8E18@liveoaksf.org> In-Reply-To: <200802040900.54630.jhb@freebsd.org> References: <429F40B0-20EE-4F47-847A-A6B1E91BA79F@liveoaksf.org> <47A217FC.1080606@root.org> <8EE3D963-E390-4F45-A1D1-2295C1767B80@liveoaksf.org> <200802040900.54630.jhb@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Feb 4, 2008, at 6:00 AM, John Baldwin wrote:
> On Thursday 31 January 2008 02:35:52 pm Tech Lab Manager wrote:
>> On Jan 31, 2008, at 10:48 AM, Nate Lawson wrote:
>>
>>> Tech Lab Manager wrote:
>>>> Sorry for the cross-post from freebsd-smb.
>>>> Building 6.3-RELEASE and 7.0-RC1 on dual Xeon (4 CPU) boxes:
>>>> options SMP
>>>> device apic
>>>> SMP kernel builds fine, all 4 CPUs launch on reboot.
>>>> But I get a TON of interrupts from acpi0 -- about 67,000 per second
>>>> according to vmstat -i. With system at idle and almost no services
>>>> running, here is output of top -S:
>>>> last pid: 877; load averages: 1.18, 0.48, 0.19
>>>> 75 processes: 6 running, 54 sleeping, 15 waiting
>>>> CPU states: 0.0% user, 0.0% nice, 0.2% system, 22.4%
>>>> interrupt, 77.4% idle
>>>> Mem: 31M Active, 12M Inact, 28M Wired, 16K Cache, 15M Buf, 3822M
>>>> Free
>>>> Swap: 4096M Total, 4096M Free
>>>> PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU
>>>> COMMAND
>>>> 10 root 1 171 52 0K 8K RUN 3 1:11 99.18%
>>>> idle: cpu3
>>>> 13 root 1 171 52 0K 8K CPU0 0 1:10 98.88%
>>>> idle: cpu0
>>>> 12 root 1 171 52 0K 8K CPU1 1 1:09 98.78%
>>>> idle: cpu1
>>>> 21 root 1 -52 -171 0K 8K CPU2 2 0:54 87.24%
>>>> irq9: acpi0
>>>> 11 root 1 171 52 0K 8K RUN 2 0:17 11.19%
>>>> idle: cpu2
>>>> Notice high load and interrupt % of CPU.
>>>> If turn off ACPI (e.g. set hint.apic.0.disabled=1 in /boot/
>>>> loader.conf),
>>>> the interrupt storm ceases, but then I'm only running on one CPU.
>>>
>>> That doesn't turn off acpi, that turns of the APIC (interrupt
>>> controller). Try:
>>> hint.acpi.0.disabled=1
>>
>> Sorry, my mistake in writing ACPI above -- I *was* trying to turn off
>> apic, based on a note in the FreeBSD handbook.
>>
>> Disabling ACPI as you suggest above has the same effect as turning
>> off APIC: the interrupt storm is disabled but only one CPU is
>> launched.
>>
>>>
>>>> The BIOS ACPI settings are all Enabled. Hyperthreading is Enabled.
>>>> These machines have been running RedHat Enterprise 5.0 with full
>>>> multiprocessor support.
>>>
>>> This looks like a failure to sleep in C1 (hlt). Someone else
>>> reported this probably earlier, but all debugging showed the
>>> inexplicable -- the HLT instruction was being executed but just did
>>> not work (returned immediately).
>>>
>>> There will be a new 7.0 build that fixes one interrupt storm
>>> related to level-triggered GPEs. If you can cvsup your 7.0 branch
>>> (RELENG_7_0) and retry, that might be helpful to see if it also
>>> fixes your problem.
>>
>> okay, I'm on RC1, will switch to RELENG and report back.
>>
>> I'm not sure if this is a red herring, but acpidump -t reports:
>>
>> Type=INT Override
>> BUS=0
>> IRQ=0
>> INTR=2
>> Flags={Polarity=conforming, Trigger=conforming}
>>
>> which looks wrong on several counts (IRQ, INTR should be 9,
>> Trigger=level). dmesg even says:
>> "MADT: Forcing active-low polarity and level trigger for SCI"
>
> No, this is an entry for something other than the SCI. You can
> have multiple
> interrupt override entries and this entry is typical on all x86
> systems with
> APICs (the 8259As are tied into pin 0 as a daisy chain and IRQ0 is
> tied into
> intpin 2 since IRQ2 isn't usable with 8259As. Do you have an entry
> at all
> for IRQ 9? If not, then the hw.acpi.sci tunables currently won't
> do anything
> (I can fix it so that they do, however).
>
Gotcha. (I'm really not much of a hardware guy so bear with me here.
note: I csup'ed the RELENG_7_0 branch last Friday but no luck on this
issue yet.)
Anyway, I see no IRQ 9 entry in acpidump, which I guess would explain
why the attempt to set SCI tunables has so apparent effect.
# acpidump -t |less
/*
RSD PTR: OEM=ACPIAM, ACPI_Rev=1.0x (0)
RSDT=0xf7ff0000, cksum=64
*/
/*
RSDT: Length=48, Revision=1, Checksum=38,
OEMID=A M I, OEM Table ID=OEMRSDT, OEM Revision=0x4000308,
Creator ID=MSFT, Creator Revision=0x97
Entries={ 0xf7ff0200, 0xf7ff0300, 0xf7fff040 }
*/
/*
FACP: Length=129, Revision=2, Checksum=65,
OEMID=A M I, OEM Table ID=OEMFACP, OEM Revision=0x4000308,
Creator ID=MSFT, Creator Revision=0x97
FACS=0xf7fff000, DSDT=0xf7ff03a0
INT_MODEL=APIC
Preferred_PM_Profile=Unspecified (0)
SCI_INT=9
SMI_CMD=0xb2, ACPI_ENABLE=0xe1, ACPI_DISABLE=0x1e,
S4BIOS_REQ=0x0
PSTATE_CNT=0xe2
PM1a_EVT_BLK=0x400-0x403
PM1a_CNT_BLK=0x404-0x405
PM2_CNT_BLK=0x420-0x420
PM_TMR_BLK=0x408-0x40b
GPE0_BLK=0x428-0x42b
GPE1_BLK=0x42c-0x42f, GPE1_BASE=16
CST_CNT=0xe3
P_LVL2_LAT=101 us, P_LVL3_LAT=1001 us
FLUSH_SIZE=1024, FLUSH_STRIDE=16
DUTY_OFFSET=1, DUTY_WIDTH=3
DAY_ALRM=13, MON_ALRM=0, CENTURY=0
IAPC_BOOT_ARCH={LEGACY_DEV,8042}
Flags={WBINVD,PROC_C1,P_LVL2_UP,SLP_BUTTON,RTC_S4,HEADLESS}
*/
/*
FACS: Length=64, HwSig=0x00000000, Firm_Wake_Vec=0x00000000
Global_Lock=
Flags=
Version=1
*/
/*
DSDT: Length=12275, Revision=1, Checksum=151,
OEMID=0ABBP, OEM Table ID=0ABBP000, OEM Revision=0x0,
Creator ID=INTL, Creator Revision=0x2002026
*/
/*
APIC: Length=146, Revision=1, Checksum=145,
OEMID=A M I, OEM Table ID=OEMAPIC, OEM Revision=0x4000308,
Creator ID=MSFT, Creator Revision=0x97
Local APIC ADDR=0xfee00000
Flags={PC-AT}
Type=Local APIC
ACPI CPU=1
Flags={ENABLED}
APIC ID=0
Type=Local APIC
ACPI CPU=2
Flags={ENABLED}
APIC ID=6
Type=Local APIC
ACPI CPU=3
Flags={ENABLED}
APIC ID=1
Type=Local APIC
ACPI CPU=4
Flags={ENABLED}
APIC ID=7
Type=IO APIC
APIC ID=8
INT BASE=0
ADDR=0x00000000fec00000
Type=IO APIC
APIC ID=9
INT BASE=24
ADDR=0x00000000fec80000
Type=IO APIC
APIC ID=10
INT BASE=48
ADDR=0x00000000fec80400
Type=IO APIC
APIC ID=11
INT BASE=72
ADDR=0x00000000fec81000
Type=IO APIC
APIC ID=12
INT BASE=96
ADDR=0x00000000fec81400
Type=INT Override
BUS=0
IRQ=0
INTR=2
Flags={Polarity=conforming, Trigger=conforming}
*/
/*
OEMB: Length=63, Revision=1, Checksum=73,
OEMID=A M I, OEM Table ID=OEMBIOS, OEM Revision=0x4000308,
Creator ID=MSFT, Creator Revision=0x97
*/
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1756A487-5F0B-466D-921D-B9F6B6FF8E18>