From owner-freebsd-smp@FreeBSD.ORG Mon Jan 28 23:25:43 2008 Return-Path: Delivered-To: freebsd-smp@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2BA2916A47D for ; Mon, 28 Jan 2008 23:25:43 +0000 (UTC) (envelope-from tlambert2@mindspring.com) Received: from elasmtp-masked.atl.sa.earthlink.net (elasmtp-masked.atl.sa.earthlink.net [209.86.89.68]) by mx1.freebsd.org (Postfix) with ESMTP id F0B6513C4EE for ; Mon, 28 Jan 2008 23:25:42 +0000 (UTC) (envelope-from tlambert2@mindspring.com) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=dk20050327; d=mindspring.com; b=sWT33lwJhisraJlz1OHsTkCibB26bAAxPjcbmFEYWBnyyfLqhlEbT9t6se66AUf6; h=Message-ID:Date:From:Reply-To:To:Subject:Mime-Version:Content-Type:Content-Transfer-Encoding:X-Mailer:X-ELNK-Trace:X-Originating-IP; Received: from [209.86.224.29] (helo=mswamui-cedar.atl.sa.earthlink.net) by elasmtp-masked.atl.sa.earthlink.net with asmtp (Exim 4.34) id 1JJctz-0007Sy-Kt; Mon, 28 Jan 2008 17:56:32 -0500 Received: from 17.202.40.209 by webmail.atl.earthlink.net with HTTP; Mon, 28 Jan 2008 17:56:31 -0500 Message-ID: <23051453.1201560991679.JavaMail.root@mswamui-cedar.atl.sa.earthlink.net> Date: Mon, 28 Jan 2008 14:56:31 -0800 (GMT-08:00) From: Terry Lambert To: "Frank W. Miller" , freebsd-smp@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Mailer: EarthLink Zoo Mail 1.0 X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a485dff743125ee44b567e2687f8975dbb2601a10902912494350badd9bab72f9c350badd9bab72f9c X-Originating-IP: 209.86.224.29 Cc: Subject: Re: intel multi-core support X-BeenThere: freebsd-smp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Terry Lambert List-Id: FreeBSD SMP implementation group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Jan 2008 23:25:43 -0000 Frank W. Miller wrote: >Greetings, > >I've been scanning the source code tree. Can anyone point me at the files >where code for enabling and managing Intel multi-core functionality is >located? They are built into the ACPI table in your BIOS, so the OS "just finds them" when it reads the processor SMP table information from there. If you are installing a multicore CPU in a motherboard that's not expecting it, you will need to update your BIOS at the same time so that the processor is reported to the OS. -- Terry From owner-freebsd-smp@FreeBSD.ORG Tue Jan 29 04:24:19 2008 Return-Path: Delivered-To: freebsd-smp@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5750516A418 for ; Tue, 29 Jan 2008 04:24:19 +0000 (UTC) (envelope-from fwmiller@cornfed.com) Received: from hiltonsmtp.worldspice.net (hiltonsmtp.worldspice.net [216.37.94.58]) by mx1.freebsd.org (Postfix) with ESMTP id E739813C43E for ; Tue, 29 Jan 2008 04:24:18 +0000 (UTC) (envelope-from fwmiller@cornfed.com) Received: (qmail 398 invoked by uid 0); 29 Jan 2008 03:57:37 -0000 Received: by simscan 1.2.0 ppid: 341, pid: 384, t: 2.0221s scanners: clamav: 0.91.2/m: spam: 3.2.3 X-Spam-Checker-Version: SpamAssassin 3.2.4 (2008-01-01) on hiltonsmtp.worldspice.net X-Spam-Level: X-Spam-Status: No, score=-2.6 Bayes=0.0003 required=6.5 autolearn=ham version=3.2.4 report= * -2.6 BAYES_00 BODY: Bayesian spam probability is 0 to 1% * [score: 0.0003] Received: from unknown (HELO cornfed) (fwmiller@cornfed.com@12.189.100.2) by hiltonsmtp.worldspice.net with ESMTPA; 29 Jan 2008 03:57:35 -0000 From: "Frank W. Miller" To: "'Terry Lambert'" , Date: Mon, 28 Jan 2008 20:57:23 -0700 Message-ID: <008d01c8622b$1579f8d0$08ac1cac@cornfed> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.2627 In-Reply-To: <23051453.1201560991679.JavaMail.root@mswamui-cedar.atl.sa.earthlink.net> X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2900.3198 Importance: Normal Cc: Subject: RE: intel multi-core support X-BeenThere: freebsd-smp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: FreeBSD SMP implementation group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Jan 2008 04:24:19 -0000 What I'm more interested in is the code that the boot cpu uses to kick the other cpus and the code that the other cpus execute to initialize themselves once they've been kicked. Thanks, FM -----Original Message----- From: Terry Lambert [mailto:tlambert2@mindspring.com] Sent: Monday, January 28, 2008 3:57 PM To: Frank W. Miller; freebsd-smp@freebsd.org Subject: Re: intel multi-core support Frank W. Miller wrote: >Greetings, > >I've been scanning the source code tree. Can anyone point me at the files >where code for enabling and managing Intel multi-core functionality is >located? They are built into the ACPI table in your BIOS, so the OS "just finds them" when it reads the processor SMP table information from there. If you are installing a multicore CPU in a motherboard that's not expecting it, you will need to update your BIOS at the same time so that the processor is reported to the OS. -- Terry From owner-freebsd-smp@FreeBSD.ORG Wed Jan 30 18:11:23 2008 Return-Path: Delivered-To: freebsd-smp@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A4EA516A418 for ; Wed, 30 Jan 2008 18:11:23 +0000 (UTC) (envelope-from tech@liveoaksf.org) Received: from assassin.liveoaksf.org (mail.liveoaksf.org [216.31.235.91]) by mx1.freebsd.org (Postfix) with SMTP id 868A213C455 for ; Wed, 30 Jan 2008 18:11:23 +0000 (UTC) (envelope-from tech@liveoaksf.org) Received: (qmail 13644 invoked by uid 1004); 30 Jan 2008 17:45:27 -0000 Received: from 192.168.1.45 by assassin.liveoaksf.org (envelope-from , uid 1002) with qmail-scanner-1.25-st-qms (clamdscan: 0.92/5575. spamassassin: 3.2.4. perlscan: 1.25-st-qms. Clear:RC:0(192.168.1.45):SA:0(-4.3/4.5):. Processed in 2.339664 secs); 30 Jan 2008 17:45:27 -0000 X-Spam-Status: No, hits=-4.3 required=4.5 X-Antivirus-LIVEOAKSF-Mail-From: tech@liveoaksf.org via assassin.liveoaksf.org X-Antivirus-LIVEOAKSF: 1.25-st-qms (Clear:RC:0(192.168.1.45):SA:0(-4.3/4.5):. Processed in 2.339664 secs Process 13637) Received: from unknown (HELO ?192.168.1.45?) (tech@liveoaksf.org@192.168.1.45) by assassin.liveoaksf.org with SMTP; 30 Jan 2008 17:45:25 -0000 Mime-Version: 1.0 (Apple Message framework v753) Content-Transfer-Encoding: 7bit Message-Id: <7D3DA805-4C15-48D1-939C-ECC6D80A014E@liveoaksf.org> Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed To: freebsd-smp@freebsd.org From: Tech Lab Manager Date: Wed, 30 Jan 2008 09:44:38 -0800 X-Mailer: Apple Mail (2.753) Subject: SMP and interrupt storm X-BeenThere: freebsd-smp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: FreeBSD SMP implementation group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jan 2008 18:11:23 -0000 Hello, Sorry for the semi-cross-post from freebsd-questions but I didn't get any bites there. Building 6.3-RELEASE on dual Xeon (4 CPU) boxes: options SMP device apic SMP kernel builds fine, all 4 CPUs launch on reboot. But I get a TON of interrupts from acpi0 -- about 67,000 per second according to vmstat -i. With system at idle with almost no services running, here is output of top -S: last pid: 877; load averages: 1.18, 0.48, 0.19 75 processes: 6 running, 54 sleeping, 15 waiting CPU states: 0.0% user, 0.0% nice, 0.2% system, 22.4% interrupt, 77.4% idle Mem: 31M Active, 12M Inact, 28M Wired, 16K Cache, 15M Buf, 3822M Free Swap: 4096M Total, 4096M Free PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 10 root 1 171 52 0K 8K RUN 3 1:11 99.18% idle: cpu3 13 root 1 171 52 0K 8K CPU0 0 1:10 98.88% idle: cpu0 12 root 1 171 52 0K 8K CPU1 1 1:09 98.78% idle: cpu1 21 root 1 -52 -171 0K 8K CPU2 2 0:54 87.24% irq9: acpi0 11 root 1 171 52 0K 8K RUN 2 0:17 11.19% idle: cpu2 Notice high load and interrupt %. Same thing happened with 7.0-RELEASE_BETA4. If turn off ACPI (set hint.apic.0.disabled=1 in /boot/loader.conf), the interrupt storm ceases, but then I'm only running on one CPU. The BIOS ACPI settings are all Enabled. Hyperthreading is Enabled. These machines have been running RedHat Enterprise 5.0 with full multiprocessor support. Any pointers? TIA. -- John -- John Berliner Live Oak School http://www.liveoaksf.org From owner-freebsd-smp@FreeBSD.ORG Wed Jan 30 23:32:42 2008 Return-Path: Delivered-To: freebsd-smp@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 293B116A418 for ; Wed, 30 Jan 2008 23:32:42 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from speedfactory.net (mail.speedfactory.net [66.23.216.219]) by mx1.freebsd.org (Postfix) with ESMTP id A463213C442 for ; Wed, 30 Jan 2008 23:32:41 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from server.baldwin.cx (unverified [66.23.211.162]) by speedfactory.net (SurgeMail 3.8s) with ESMTP id 230270241-1834499 for multiple; Wed, 30 Jan 2008 18:15:10 -0500 Received: from localhost.corp.yahoo.com (john@localhost [127.0.0.1]) (authenticated bits=0) by server.baldwin.cx (8.14.2/8.14.2) with ESMTP id m0UNGEPf011699; Wed, 30 Jan 2008 18:16:20 -0500 (EST) (envelope-from jhb@freebsd.org) From: John Baldwin To: freebsd-smp@freebsd.org Date: Wed, 30 Jan 2008 18:16:10 -0500 User-Agent: KMail/1.9.7 References: <7D3DA805-4C15-48D1-939C-ECC6D80A014E@liveoaksf.org> In-Reply-To: <7D3DA805-4C15-48D1-939C-ECC6D80A014E@liveoaksf.org> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200801301816.10279.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (server.baldwin.cx [127.0.0.1]); Wed, 30 Jan 2008 18:16:23 -0500 (EST) X-Virus-Scanned: ClamAV 0.91.2/5617/Wed Jan 30 15:47:28 2008 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=4.2 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.3 X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on server.baldwin.cx Cc: Tech Lab Manager Subject: Re: SMP and interrupt storm X-BeenThere: freebsd-smp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: FreeBSD SMP implementation group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jan 2008 23:32:42 -0000 On Wednesday 30 January 2008 12:44:38 pm Tech Lab Manager wrote: > Hello, > > Sorry for the semi-cross-post from freebsd-questions but I didn't get > any bites there. > > Building 6.3-RELEASE on dual Xeon (4 CPU) boxes: > > options SMP > device apic > > SMP kernel builds fine, all 4 CPUs launch on reboot. > But I get a TON of interrupts from acpi0 -- about 67,000 per second > according to vmstat -i. With system at idle with almost no services > running, here is output of top -S: > > last pid: 877; load averages: 1.18, 0.48, 0.19 > 75 processes: 6 running, 54 sleeping, 15 waiting > CPU states: 0.0% user, 0.0% nice, 0.2% system, 22.4% interrupt, > 77.4% idle > Mem: 31M Active, 12M Inact, 28M Wired, 16K Cache, 15M Buf, 3822M Free > Swap: 4096M Total, 4096M Free > > PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU > COMMAND > 10 root 1 171 52 0K 8K RUN 3 1:11 99.18% > idle: cpu3 > 13 root 1 171 52 0K 8K CPU0 0 1:10 98.88% > idle: cpu0 > 12 root 1 171 52 0K 8K CPU1 1 1:09 98.78% > idle: cpu1 > 21 root 1 -52 -171 0K 8K CPU2 2 0:54 87.24% > irq9: acpi0 > 11 root 1 171 52 0K 8K RUN 2 0:17 11.19% > idle: cpu2 > > Notice high load and interrupt %. > Same thing happened with 7.0-RELEASE_BETA4. > > If turn off ACPI (set hint.apic.0.disabled=1 in /boot/loader.conf), > the interrupt storm ceases, but then I'm only running on one CPU. > > The BIOS ACPI settings are all Enabled. Hyperthreading is Enabled. > These machines have been running RedHat Enterprise 5.0 with full > multiprocessor support. > > Any pointers? TIA. > -- John Try freebsd-acpi@ perhaps. Are there any other devices on IRQ 9 besides acpi0? You can also try messing with the trigger mode of the ACPI interrupt via tunables IIRC (i.e. maybe it is set to edge and needs to be level). You can use the hw.acpi.sci.polarity ("hi" or "low") and hw.acpi.sci.trigger ("edge" or "level") to tweak this. You can use 'acpidump -t' to see if you have interrupt override entry for IRQ 9 to see if it's set to do the wrong thing. -- John Baldwin