From owner-freebsd-current@FreeBSD.ORG Tue Jun 15 17:55:26 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7C61A16A506; Tue, 15 Jun 2004 17:55:26 +0000 (GMT) Received: from mail.sandvine.com (sandvine.com [199.243.201.138]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6D9FF43D53; Tue, 15 Jun 2004 17:55:25 +0000 (GMT) (envelope-from don@sandvine.com) Received: by mail.sandvine.com with Internet Mail Service (5.5.2657.72) id ; Tue, 15 Jun 2004 13:55:17 -0400 Message-ID: From: Don Bowman To: "'current@freebsd.org'" , "'acpi@freebsd.org'" Date: Tue, 15 Jun 2004 13:55:13 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2657.72) Content-Type: multipart/mixed; boundary="----_=_NextPart_000_01C45301.E8603B70" Subject: acpi interrupt storm when disabling, IPI hang X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jun 2004 17:55:26 -0000 This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_000_01C45301.E8603B70 Content-Type: text/plain; charset="iso-8859-1" When i run 'acpiconf -d', i get an interrupt storm on irq9... vmstat -i shows ~1000/s. Jun 15 13:41:33 cdata kernel: Interrupt storm detected on "irq9: acpi0"; throttling interrupt source when i try to re-enable, i get an error: # acpiconf -e acpi0: interrupt handler already installed ACPI-0210: *** Error: Unable to install System Control Interrupt Handler, AE_ALREADY_EXISTS acpiconf: enable failed: Device not configured # System is a Supermicro X5DPE motherboard, dual 2.8GHz Xeon, 533MHz FSB, Intel e7501 chipset. # sysctl hw.acpi hw.acpi.supported_sleep_state: S1 S4 S5 hw.acpi.power_button_state: S5 hw.acpi.sleep_button_state: S1 hw.acpi.lid_switch_state: NONE hw.acpi.standby_state: S1 hw.acpi.suspend_state: S3 hw.acpi.sleep_delay: 1 hw.acpi.s4bios: 0 hw.acpi.verbose: 0 hw.acpi.disable_on_poweroff: 0 hw.acpi.reset_video: 1 hw.acpi.cpu.cx_supported: C1/0 hw.acpi.cpu.cx_lowest: C1 hw.acpi.cpu.cx_usage: 99% The system will not boot 5.x if acpi is disabled. It will run 4.x without trouble tho. I am currently debugging a problem where 2 of the processors are in acpi_cpu_c1, 1 is in idle, and 1 calls smp_tlb_shootdown, but the IPI to one of the other processors never completes, so it hangs. I have a core file from this state. (kgdb) printStack2 0xed09acdc : 0xa10cc483 : 0xf689c3c9 : 0xe800768d : 0xb808c483 : 0xe90cc483 (kgdb) printStack2 0xed09dcd8 : 0x5590c3c9 : 0x0000d4e9 : 0xf689c3c9 : 0xe800768d : 0xb808c483 : 0xe90cc483 (kgdb) printStack2 0xed0a0cd8 : 0x5590c3c9 : 0x0000d4e9 : 0xf689c3c9 : 0xe800768d : 0xb808c483 : 0xe90cc483 (kgdb) printStack2 0xeec7dc3c : 0x0128978b : 0x000350e9 : 0x6404c483 : 0x8d04c483 : 0x5bf8658d : 0x8d04c483 : 0xc483f689 : 0xeb04c483 : 0xb808c483 : 0xe90cc483 Sometimes that last one is like: #7 0xc068e066 in smp_tlb_shootdown (vector=246, addr1=0, addr2=0) at machine/cpufunc.h:305 #8 0xc068e1d0 in smp_invlpg_range (addr1=3876638720, addr2=3876769792) at /usr/src/sys/i386/i386/mp_machdep.c:1030 #9 0xc0690643 in pmap_invalidate_range (pmap=0xc077dd80, sva=3876638720, eva=3876769792) at /usr/src/sys/i386/i386/pmap.c:640 #10 0xc0690be0 in pmap_qenter (sva=3876638720, m=0xde548600, count=0) at /usr/src/sys/i386/i386/pmap.c:958 #11 0xc058aca3 in cluster_rbuild (vp=0xc83db104, filesize=1073741824, lbn=40844, blkno=241691456, size=16384, run=8, fbp=0x0) at /usr/src/sys/kern/vfs_cluster.c:508 #12 0xc058a3a8 in cluster_read (vp=0xc83db104, filesize=1073741824, lblkno=40844, size=16384, cred=0x0, totread=8192, seqcount=127, bpp=0x0) at /usr/src/sys/kern/vfs_cluster.c:263 #13 0xc063dc4a in ffs_read (ap=0x0) at /usr/src/sys/ufs/ffs/ffs_vnops.c:441 #14 0xc059e325 in vn_read (fp=0xc5a02198, uio=0xf53c1c88, active_cred=0xc59f6280, flags=0, td=0xc6645e70) at vnode_if.h:398 #15 0xc05655bc in dofileread (td=0xc6645e70, fp=0xc5a02198, fd=23, buf=0x706b40c0, nbyte=8192, offset=0, flags=0) at /usr/src/sys/sys/file.h:233 #16 0xc0565467 in read (td=0xc6645e70, uap=0xf53c1d14) at /usr/src/sys/kern/sys_generic.c:107 #17 0xc06956f7 in syscall (frame= {tf_fs = 138149935, tf_es = 138149935, tf_ds = -1078001617, tf_edi = 81682, tf_esi = 137404728, tf_ebp = -1077946040, tf_isp = -180609676, tf_ebx = 608, tf_edx = 23, tf_ecx = 137834496, tf_eax = 3, tf_trapno = 22, tf_err = 2, tf_eip = 1749868123, tf_cs = 31, tf_eflags = 582, tf_esp = -1077946068, tf_ss = 47}) at /usr/src/sys/i386/i386/trap.c:1004 #18 0x684cde5b in ?? () (kgdb) disass acpi_cpu_c1 Dump of assembler code for function acpi_cpu_c1: 0xc084cc44 : push %ebp 0xc084cc45 : mov %esp,%ebp 0xc084cc47 : sti 0xc084cc48 : hlt 0xc084cc49 : leave 0xc084cc4a : ret 0xc084cc4b : nop End of assembler dump. (kgdb) disass acpi_cpu_idle Dump of assembler code for function acpi_cpu_idle: 0xc084caa0 : push %ebp ... 0xc084cb4f : cltd 0xc084cb50 : idivl 0xc0753f0c 0xc084cb56 : mov %eax,0x9c(%esi) 0xc084cb5c : call 0xc084cc44 0xc084cb61 : jmp 0xc084cc3a 0xc084cb66 : mov %esi,%esi 0xc084cb68 : cmpl $0x3,0x4(%edi) 0xc084cb6c : jne 0xc084cb88 ------_=_NextPart_000_01C45301.E8603B70 Content-Type: application/octet-stream; name="dmesg.boot" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="dmesg.boot" Copyright (c) 1992-2004 The FreeBSD Project.=0A= Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, = 1994=0A= The Regents of the University of California. All rights reserved.=0A= FreeBSD 5.2-CURRENT #10: Thu Jun 10 08:54:08 EDT 2004=0A= root@cdata.sandvine.com:/usr/obj/usr/src/sys/CDATA=0A= Preloaded elf kernel "/boot/kernel/kernel" at 0xc086b000.=0A= Preloaded elf module "/boot/kernel/acpi.ko" at 0xc086b30c.=0A= ACPI APIC Table: =0A= Timecounter "i8254" frequency 1193182 Hz quality 0=0A= CPU: Intel(R) Xeon(TM) CPU 2.80GHz (2799.22-MHz 686-class CPU)=0A= Origin =3D "GenuineIntel" Id =3D 0xf27 Stepping =3D 7=0A= = Features=3D0xbfebfbff=0A= Hyperthreading: 2 logical CPUs=0A= real memory =3D 4026007552 (3839 MB)=0A= avail memory =3D 3940634624 (3758 MB)=0A= FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs=0A= cpu0 (BSP): APIC ID: 0=0A= cpu1 (AP): APIC ID: 1=0A= cpu2 (AP): APIC ID: 6=0A= cpu3 (AP): APIC ID: 7=0A= Pentium 4 TCC support enabled, current performance 100%=0A= ioapic0 irqs 0-23 on motherboard=0A= ioapic1 irqs 24-47 on motherboard=0A= ioapic2 irqs 48-71 on motherboard=0A= ioapic3 irqs 72-95 on motherboard=0A= ioapic4 irqs 96-119 on motherboard=0A= random: =0A= Pentium Pro MTRR support enabled=0A= npx0: [FAST]=0A= npx0: on motherboard=0A= npx0: INT 16 interface=0A= acpi0: on motherboard=0A= acpi0: [GIANT-LOCKED]=0A= pcibios: BIOS version 2.10=0A= acpi0: Power Button (fixed)=0A= Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000=0A= acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on = acpi0=0A= cpu0: on acpi0=0A= cpu1: on acpi0=0A= cpu2: on acpi0=0A= cpu3: on acpi0=0A= pcib0: port 0xcf8-0xcff on acpi0=0A= pci0: on pcib0=0A= pci0: at device 0.1 (no driver attached)=0A= pcib1: at device 2.0 on pci0=0A= pcib1: could not get PCI interrupt routing table for \\_SB_.PCI0.HLB_ - = AE_NOT_FOUND=0A= pci1: on pcib1=0A= pci1: at device 28.0 (no driver = attached)=0A= pcib2: at device 29.0 on pci1=0A= pci2: on pcib2=0A= pci1: at device 30.0 (no driver = attached)=0A= pcib3: at device 31.0 on pci1=0A= pci3: on pcib3=0A= em0: port = 0x3000-0x303f mem 0xf4200000-0xf421ffff irq 28 at device 2.0 on pci3=0A= em0: [GIANT-LOCKED]=0A= em0: Ethernet address: 00:30:48:2a:a0:1c=0A= em0: Speed:N/A Duplex:N/A=0A= em1: port = 0x3040-0x307f mem 0xf4220000-0xf423ffff irq 29 at device 2.1 on pci3=0A= em1: [GIANT-LOCKED]=0A= em1: Ethernet address: 00:30:48:2a:a0:1d=0A= em1: Speed:N/A Duplex:N/A=0A= pci0: at device 2.1 (no driver attached)=0A= pcib4: at device 3.0 on pci0=0A= pcib4: could not get PCI interrupt routing table for \\_SB_.PCI0.HLC_ - = AE_NOT_FOUND=0A= pci4: on pcib4=0A= pci4: at device 28.0 (no driver = attached)=0A= pcib5: at device 29.0 on pci4=0A= pci5: on pcib5=0A= aac0: mem 0xf8000000-0xfbffffff irq 96 at = device 1.0 on pci5=0A= aac0: [FAST]=0A= aac0: i960RX 100MHz, 48MB cache memory, optional battery present=0A= aac0: Kernel 4.0-0, Build 6011, S/N ba8ff3=0A= aac0: Supported = Options=3D1f7e=0A= aacp0: on aac0=0A= aacp1: on aac0=0A= pci4: at device 30.0 (no driver = attached)=0A= pcib6: at device 31.0 on pci4=0A= pci6: on pcib6=0A= pci0: at device 3.1 (no driver attached)=0A= uhci0: port = 0x2000-0x201f irq 16 at device 29.0 on pci0=0A= uhci0: [GIANT-LOCKED]=0A= usb0: on uhci0=0A= usb0: USB revision 1.0=0A= uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1=0A= uhub0: 2 ports with 2 removable, self powered=0A= uhci1: port = 0x2020-0x203f irq 19 at device 29.1 on pci0=0A= uhci1: [GIANT-LOCKED]=0A= usb1: on uhci1=0A= usb1: USB revision 1.0=0A= uhub1: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1=0A= uhub1: 2 ports with 2 removable, self powered=0A= uhci2: port = 0x2040-0x205f irq 18 at device 29.2 on pci0=0A= uhci2: [GIANT-LOCKED]=0A= usb2: on uhci2=0A= usb2: USB revision 1.0=0A= uhub2: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1=0A= uhub2: 2 ports with 2 removable, self powered=0A= pcib7: at device 30.0 on pci0=0A= pci7: on pcib7=0A= pci7: at device 1.0 (no driver attached)=0A= isab0: at device 31.0 on pci0=0A= isa0: on isab0=0A= atapci0: port = 0x2080-0x208f,0x376,0x170-0x177,0x3f6,0x1f0-0x1f7 at device 31.1 on = pci0=0A= ata0: at 0x1f0 irq 14 on atapci0=0A= ata1: at 0x170 irq 15 on atapci0=0A= ichsmb0: port 0x1100-0x111f irq = 17 at device 31.3 on pci0=0A= ichsmb0: [GIANT-LOCKED]=0A= smbus0: on ichsmb0=0A= smb0: on smbus0=0A= acpi_button0: on acpi0=0A= atkbdc0: port 0x64,0x60 irq 1 on acpi0=0A= atkbd0: flags 0x1 irq 1 on atkbdc0=0A= kbd0 at atkbd0=0A= atkbd0: [GIANT-LOCKED]=0A= sio0 port 0x3f8-0x3ff irq 4 on acpi0=0A= sio0: type 16550A, console=0A= sio1 port 0x2f8-0x2ff irq 3 on acpi0=0A= sio1: type 16550A=0A= fdc0: port = 0x3f7,0x3f0-0x3f5 irq 6 drq 2 on acpi0=0A= fdc0: FIFO enabled, 8 bytes threshold=0A= orm0: