From owner-freebsd-current@FreeBSD.ORG Sun May 21 11:16:20 2006 Return-Path: X-Original-To: current@freebsd.org Delivered-To: freebsd-current@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A666E16A420 for ; Sun, 21 May 2006 11:16:20 +0000 (UTC) (envelope-from joseph.koshy@gmail.com) Received: from wx-out-0102.google.com (wx-out-0102.google.com [66.249.82.194]) by mx1.FreeBSD.org (Postfix) with ESMTP id DAEA943D49 for ; Sun, 21 May 2006 11:16:19 +0000 (GMT) (envelope-from joseph.koshy@gmail.com) Received: by wx-out-0102.google.com with SMTP id i31so751624wxd for ; Sun, 21 May 2006 04:16:19 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=gSeiCp77z8fRcU+cBiECh0W7EOU2M2OHTL3QpHqBBG9S1cJkv/ak+okKANzEMODCkwZkBS/Q1/Bnppgx0l7jDWCC4SGgiduz/r1YZsXR/X1dOGd0y09g+tliMvVNILraMuFHb/86KAS68w/7GNQC4VPxs4u13DDJ5FJg2LLbVIA= Received: by 10.70.46.9 with SMTP id t9mr1554860wxt; Sun, 21 May 2006 04:16:19 -0700 (PDT) Received: by 10.70.105.1 with HTTP; Sun, 21 May 2006 04:16:19 -0700 (PDT) Message-ID: <84dead720605210416y268f890end72848a9edc293b7@mail.gmail.com> Date: Sun, 21 May 2006 16:46:19 +0530 From: "Joseph Koshy" To: "Robert Watson" In-Reply-To: <20060520233201.B8068@fledge.watson.org> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable Content-Disposition: inline References: <20060520233201.B8068@fledge.watson.org> Cc: current@freebsd.org Subject: Re: PMC enters debugger with NMI on ctrl-C X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 May 2006 11:16:20 -0000 > tiger-2# pmcstat -S unhalted-cycles -O /tmp/sample.out > ^CNMI ... going to debugger > [thread pid 898 tid 100175 ] > Stopped at p4_stop_pmc+0x70: movl $0,%eax > db> bt > Tracing pid 898 tid 100175 td 0xc765c360 > p4_stop_pmc(0,1,1,0,0) at p4_stop_pmc+0x70 > pmc_release_pmc_descriptor(c703ca00,c06c03f3,c1032860,c103cac8,e992fb18) = at > pmc_release_pmc_descriptor+0x61 > pmc_syscall_handler(c765c360,e992fd04,2,202,c09c07d8) at > pmc_syscall_handler+0xf6f > syscall(3b,3b,bfbf003b,0,8050ee0) at syscall+0x2ee > Xint0x80_syscall() at Xint0x80_syscall+0x1f > --- syscall (210, FreeBSD ELF32, pmc_syscall_handler), eip =3D 0x280d104d= , esp =3D > 0xbfbfe6e0, ebp =3D 0xbfbfe6f8 --- > db> > I'll leave it in the debugger overnight in case there's > anything useful to be done with it. There appears to be a race here and here is my guess as to how this is getting triggered. 0. An NMI is posted by the PMC and is in the process of working its way through the processors innards. 1. In the meantime, at p4_stop_pmc+0x6e, the CPU turns off the interrupting PMC by turning off its enable bit with a WRMSR instruction. This also zeroes the CCCR_OVF bits in that register. 2. The processor now takes its NMI interrupt at the boundary of the WRMSR instruction (at p4_stop_pmc+6e). The PMC handler p4_intr() doesn't find any PMC with a CCCR_OVF bit set and so assumes that the NMI wasn't caused by a PMC. It bounces the NMI to trap() which promptly panics. I need to think about this. Here is a work-around for now: Index: hwpmc_piv.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /cvs/FreeBSD/src/sys/dev/hwpmc/hwpmc_piv.c,v retrieving revision 1.13 diff -u -u -r1.13 hwpmc_piv.c --- hwpmc_piv.c 28 Mar 2006 14:09:21 -0000 1.13 +++ hwpmc_piv.c 21 May 2006 11:02:26 -0000 @@ -1698,7 +1698,7 @@ atomic_add_int(did_interrupt ? &pmc_stats.pm_intr_processed : &pmc_stats.pm_intr_ignored, 1); - return did_interrupt; + return 1; } > BTW, saw some oddness. I capture the PMC samples on one > box, and post-process on another. This results in the > following oddness: I used the above pmcstat command to > track unhalted-cycles on a Dual Xeon, then post-processed > on an amd64 box, so pmcstat generated gmon output with > the name p4-global-power-events. Perhaps pmcstat should > capture the event name in its data file so that when doing > later post-processing, it can use the names from the > machine the captures were on, rather than the names of the > machine the processing is being done on? It does this already. 'unhalted-cycles' is an alias that is converted to the machine specific PMC name at the time of data collection. Intel P4 Xeons alias unhalted-cycles to event 'p4-global-power-events'. On an amd64 'unhalted-cycles' maps to 'k8-bu-cpu-clk-unhalted'. --=20 FreeBSD Developer, http://people.freebsd.org/~jkoshy