From owner-freebsd-acpi@FreeBSD.ORG Sun Jun 3 06:23:35 2012 Return-Path: Delivered-To: freebsd-acpi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 68A85106564A; Sun, 3 Jun 2012 06:23:35 +0000 (UTC) (envelope-from iwasaki@jp.FreeBSD.org) Received: from locore.org (ns01.locore.org [218.45.21.227]) by mx1.freebsd.org (Postfix) with ESMTP id 1AAD08FC0C; Sun, 3 Jun 2012 06:23:34 +0000 (UTC) Received: from localhost (celeron.v4.locore.org [192.168.0.10]) by locore.org (8.14.5/8.14.5/iwasaki) with ESMTP/inet id q536NQ1N029701; Sun, 3 Jun 2012 15:23:26 +0900 (JST) (envelope-from iwasaki@jp.FreeBSD.org) Date: Sun, 03 Jun 2012 15:23:26 +0900 (JST) Message-Id: <20120603.152326.105529138.iwasaki@jp.FreeBSD.org> To: attilio@freebsd.org From: Mitsuru IWASAKI In-Reply-To: References: <20120603.002554.119853142.iwasaki@jp.FreeBSD.org> X-Mailer: Mew version 3.3 on Emacs 20.7 / Mule 4.0 (HANANOEN) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-acpi@freebsd.org Subject: Re: preparation for x86/acpica/acpi_wakeup.c X-BeenThere: freebsd-acpi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ACPI and power management development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Jun 2012 06:23:35 -0000 Hi, thanks for your comments. > I still didn't review your patch in detail, but I think PCB_SUSPENDING > is not really the way it should be done. > PCB datas are about the thread state on a particular CPU and the > suspension of the CPU has not much to do with the thread state. It > should be more a CPU state itself. This area is very weak in FreeBSD > right now, however, but I will try to dump my thinking on it right > now. > > The first thing to consider is that right now we only have 2 states > for CPUs: started and stopped. These states are controlled by > started_cpus and stopped_cpus masks respectively. It seems you really > want to add an intermediate level among the 2 where you have: started > -> suspended -> started -> suspended ... -> stopped and you need to > expand the mechanism for dealing with started and stopped cpus to do > that. I'm pretty sure this will be very helpful also for other > architectures that want to do the same. You are right, I'll add new global cpuset suspended_cpus and replace PCB_SUSPENDING with it. Thanks! From owner-freebsd-acpi@FreeBSD.ORG Sun Jun 3 07:19:07 2012 Return-Path: Delivered-To: freebsd-acpi@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C561F106564A; Sun, 3 Jun 2012 07:19:07 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id A3D148FC12; Sun, 3 Jun 2012 07:19:06 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id KAA28084; Sun, 03 Jun 2012 10:19:04 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1Sb55c-000M72-5L; Sun, 03 Jun 2012 10:19:04 +0300 Message-ID: <4FCB0FE5.4050607@FreeBSD.org> Date: Sun, 03 Jun 2012 10:19:01 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:12.0) Gecko/20120503 Thunderbird/12.0.1 MIME-Version: 1.0 To: Attilio Rao , Mitsuru IWASAKI References: <20120603.002554.119853142.iwasaki@jp.FreeBSD.org> In-Reply-To: X-Enigmail-Version: 1.5pre Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: freebsd-acpi@FreeBSD.org, freebsd-arch@FreeBSD.org Subject: cpu stopping [Was: preparation for x86/acpica/acpi_wakeup.c] X-BeenThere: freebsd-acpi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ACPI and power management development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Jun 2012 07:19:07 -0000 on 03/06/2012 00:39 Attilio Rao said the following: > The first thing to consider is that right now we only have 2 states > for CPUs: started and stopped. These states are controlled by > started_cpus and stopped_cpus masks respectively. It seems you really > want to add an intermediate level among the 2 where you have: started > -> suspended -> started -> suspended ... -> stopped and you need to > expand the mechanism for dealing with started and stopped cpus to do > that. I'm pretty sure this will be very helpful also for other > architectures that want to do the same. As the first thing I must admit that I haven't looked at the patch :-) But really I don't see why we need to differentiate between stopped and suspended state as both of them ultimately mean exactly the same thing - CPUs are spinning on some condition (and they are in a well-defined place and state). My view of how this should work is: - there can be only one master CPU that controls all other (slave) CPUs - the master sets entry and exit hooks - the master signals slaves to enter the stop state - the slaves execute the enter hook and start spinning on the release condition - the master does whatever it wants to do in this special system state - the master signals the slaves to resume - the slave exit the spin loop and execute the exit hook We have almost all of this in place. Only now we have different IPIs and different IPI handlers to do the job (cpustop_handler and cpususpend_handler). I think that the hooks model should be more universal. In my opinion, what really would deserve a completely independent path is the hard-stop case. As this can be invoked nested to the other cases. E.g. exotic situations like a breakpoint or a trap or a panic in the suspend or the normal stop code paths. -- Andriy Gapon From owner-freebsd-acpi@FreeBSD.ORG Sun Jun 3 09:54:10 2012 Return-Path: Delivered-To: freebsd-acpi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6A6AC106564A; Sun, 3 Jun 2012 09:54:10 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-lpp01m010-f54.google.com (mail-lpp01m010-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id 7908D8FC08; Sun, 3 Jun 2012 09:54:09 +0000 (UTC) Received: by laai10 with SMTP id i10so3072293laa.13 for ; Sun, 03 Jun 2012 02:54:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=Pr4Hnj2HezsD1rehQuPsqCPcosSqnis1+4C86gL9b50=; b=WvH6NwwC/Ix+vXjpZx/aZxicqBDNEqCfqd2xK20qQaEu33kbIpese2sz68YUzszipm 3iC62advSifR+flumJlcijLxoann4oMUSJsWiS4aBf1ThZqh66wqr8Z1ckOfknXmrR0R 27lE/v8a/9PikykdUrdKwkIumen/4o6Al3ppoTRGfJ3rTh5QFT+LgZUsJz5153RxV5sf yOE9HT4zKJNDeaekISQHfWln1LkxLOieH/iXjzjMcnlV5c/wufDh8xutg6NdjNFcEl/G wdiwwi9aeF6F2kPIE7h5r8gLTYxgm1nj+EJS1DY4atHi1ME3wsoP/kYCKZjtY6EqlCHl fvIA== MIME-Version: 1.0 Received: by 10.152.103.11 with SMTP id fs11mr8689233lab.23.1338717248070; Sun, 03 Jun 2012 02:54:08 -0700 (PDT) Sender: asmrookie@gmail.com Received: by 10.112.27.65 with HTTP; Sun, 3 Jun 2012 02:54:07 -0700 (PDT) In-Reply-To: <4FCB0FE5.4050607@FreeBSD.org> References: <20120603.002554.119853142.iwasaki@jp.FreeBSD.org> <4FCB0FE5.4050607@FreeBSD.org> Date: Sun, 3 Jun 2012 10:54:07 +0100 X-Google-Sender-Auth: qSLZcHBUQgn9exkzIhnujbmC38s Message-ID: From: Attilio Rao To: Andriy Gapon Content-Type: text/plain; charset=UTF-8 Cc: freebsd-acpi@freebsd.org, freebsd-arch@freebsd.org Subject: Re: cpu stopping [Was: preparation for x86/acpica/acpi_wakeup.c] X-BeenThere: freebsd-acpi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ACPI and power management development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Jun 2012 09:54:10 -0000 2012/6/3 Andriy Gapon : > on 03/06/2012 00:39 Attilio Rao said the following: >> The first thing to consider is that right now we only have 2 states >> for CPUs: started and stopped. These states are controlled by >> started_cpus and stopped_cpus masks respectively. It seems you really >> want to add an intermediate level among the 2 where you have: started >> -> suspended -> started -> suspended ... -> stopped and you need to >> expand the mechanism for dealing with started and stopped cpus to do >> that. I'm pretty sure this will be very helpful also for other >> architectures that want to do the same. > > As the first thing I must admit that I haven't looked at the patch :-) > > > But really I don't see why we need to differentiate between stopped and > suspended state as both of them ultimately mean exactly the same thing - CPUs > are spinning on some condition (and they are in a well-defined place and state). This is debeatable and I'm not sure I agree. At some point we may want to implement CPU on-the-fly suspension for CPUs which is a different event than "stopping" (where stopping will be "permanent stopping" and suspending will be "possible to recover suspension"). The important thing about this is that we need to expand our model in a way that it makes simple to add more states to the CPUs than simple started/stopped. Right now we don't have any architecture for this in place. > My view of how this should work is: > - there can be only one master CPU that controls all other (slave) CPUs > - the master sets entry and exit hooks > - the master signals slaves to enter the stop state > - the slaves execute the enter hook and start spinning on the release condition > - the master does whatever it wants to do in this special system state > - the master signals the slaves to resume > - the slave exit the spin loop and execute the exit hook > > We have almost all of this in place. Only now we have different IPIs and > different IPI handlers to do the job (cpustop_handler and cpususpend_handler). > I think that the hooks model should be more universal. For hook you mean like a rendezvous handler? I'm not sure I understand otherwise. > In my opinion, what really would deserve a completely independent path is the > hard-stop case. As this can be invoked nested to the other cases. E.g. exotic > situations like a breakpoint or a trap or a panic in the suspend or the normal > stop code paths. What I'm really interested is expanding our model in a way that it can handle multiple CPU states. Then it is just a matter of adding the right states and it is all trivial work. And however, as already mentioned, I'm not sure I would assimilate suspended = stopped. Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-acpi@FreeBSD.ORG Sun Jun 3 14:42:47 2012 Return-Path: Delivered-To: freebsd-acpi@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9135A106566B; Sun, 3 Jun 2012 14:42:47 +0000 (UTC) (envelope-from iwasaki@jp.FreeBSD.org) Received: from locore.org (ns01.locore.org [218.45.21.227]) by mx1.freebsd.org (Postfix) with ESMTP id 236248FC0C; Sun, 3 Jun 2012 14:42:47 +0000 (UTC) Received: from localhost (celeron.v4.locore.org [192.168.0.10]) by locore.org (8.14.5/8.14.5/iwasaki) with ESMTP/inet id q53EgiCq031408; Sun, 3 Jun 2012 23:42:44 +0900 (JST) (envelope-from iwasaki@jp.FreeBSD.org) Date: Sun, 03 Jun 2012 23:42:43 +0900 (JST) Message-Id: <20120603.234243.28389486.iwasaki@jp.FreeBSD.org> To: avg@FreeBSD.org From: Mitsuru IWASAKI In-Reply-To: <4FCB0FE5.4050607@FreeBSD.org> References: <20120603.002554.119853142.iwasaki@jp.FreeBSD.org> <4FCB0FE5.4050607@FreeBSD.org> X-Mailer: Mew version 3.3 on Emacs 20.7 / Mule 4.0 (HANANOEN) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: attilio@FreeBSD.org, freebsd-acpi@FreeBSD.org, freebsd-arch@FreeBSD.org Subject: Re: cpu stopping X-BeenThere: freebsd-acpi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ACPI and power management development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Jun 2012 14:42:47 -0000 Hi, thanks for comments. > As the first thing I must admit that I haven't looked at the patch :-) Never mind :) What I'm trying to do in the patches is just to unify amd64/i386 independent part (acpi_wakeup.c) for the code maintenance, so please let's commit it first, then start re-design the cpususpend_handler(). > But really I don't see why we need to differentiate between stopped and > suspended state as both of them ultimately mean exactly the same thing - CPUs > are spinning on some condition (and they are in a well-defined place and state). Yes, amd64/i386 cpususpend_handler() is very similar to cpustop_handler() actually, some resume related procedures are added for suspend. > My view of how this should work is: > - there can be only one master CPU that controls all other (slave) CPUs > - the master sets entry and exit hooks Entry hook for suspending might be ---- ctx_fpusave(suspfpusave[cpu]); wbinvd(); CPU_SET_ATOMIC(cpu, &stopped_cpus); ---- and for stopping is ---- /* Indicate that we are stopped */ CPU_SET_ATOMIC(cpu, &stopped_cpus); ---- Correct? I think stopping hook can be replaced with suspending hook. Exit hook for suspending is ---- pmap_init_pat(); load_cr3(susppcbs[cpu]->pcb_cr3); initializecpu(); PCPU_SET(switchtime, 0); PCPU_SET(switchticks, ticks); [snip] /* Resume MCA and local APIC */ mca_resume(); lapic_setup(0); ---- For stopping should be ---- if (cpu == 0 && cpustop_restartfunc != NULL) { cpustop_restartfunc(); cpustop_restartfunc = NULL; } ---- > - the master signals slaves to enter the stop state > - the slaves execute the enter hook and start spinning on the release condition > - the master does whatever it wants to do in this special system state > - the master signals the slaves to resume > - the slave exit the spin loop and execute the exit hook I think it would be possible. However I personally think that priority of x86/x86/mp_machdep.c is higher and more effective than merging cpususpend/stop_handler(). Thanks From owner-freebsd-acpi@FreeBSD.ORG Sun Jun 3 14:58:43 2012 Return-Path: Delivered-To: freebsd-acpi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 68C521065674 for ; Sun, 3 Jun 2012 14:58:43 +0000 (UTC) (envelope-from iwasaki@jp.FreeBSD.org) Received: from locore.org (ns01.locore.org [218.45.21.227]) by mx1.freebsd.org (Postfix) with ESMTP id 1726C8FC17 for ; Sun, 3 Jun 2012 14:58:42 +0000 (UTC) Received: from localhost (celeron.v4.locore.org [192.168.0.10]) by locore.org (8.14.5/8.14.5/iwasaki) with ESMTP/inet id q53Eweut031477 for ; Sun, 3 Jun 2012 23:58:40 +0900 (JST) (envelope-from iwasaki@jp.FreeBSD.org) Date: Sun, 03 Jun 2012 23:58:39 +0900 (JST) Message-Id: <20120603.235839.133292778.iwasaki@jp.FreeBSD.org> To: freebsd-acpi@freebsd.org From: Mitsuru IWASAKI In-Reply-To: <20120603.152326.105529138.iwasaki@jp.FreeBSD.org> References: <20120603.002554.119853142.iwasaki@jp.FreeBSD.org> <20120603.152326.105529138.iwasaki@jp.FreeBSD.org> X-Mailer: Mew version 3.3 on Emacs 20.7 / Mule 4.0 (HANANOEN) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: Re: preparation for x86/acpica/acpi_wakeup.c X-BeenThere: freebsd-acpi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ACPI and power management development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Jun 2012 14:58:43 -0000 Hi, > > The first thing to consider is that right now we only have 2 states > > for CPUs: started and stopped. These states are controlled by > > started_cpus and stopped_cpus masks respectively. It seems you really > > want to add an intermediate level among the 2 where you have: started > > -> suspended -> started -> suspended ... -> stopped and you need to > > expand the mechanism for dealing with started and stopped cpus to do > > that. I'm pretty sure this will be very helpful also for other > > architectures that want to do the same. > > You are right, I'll add new global cpuset suspended_cpus and replace > PCB_SUSPENDING with it. Also I noticed that CR3 can be restored in resumectx() and remove load_cr3() from C code. Revised patches: http://people.freebsd.org/~iwasaki/acpi/x86-acpi_wakeup-20120603.diff Thanks From owner-freebsd-acpi@FreeBSD.ORG Sun Jun 3 19:35:26 2012 Return-Path: Delivered-To: freebsd-acpi@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 53EEA106566B; Sun, 3 Jun 2012 19:35:26 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 3964C8FC1B; Sun, 3 Jun 2012 19:35:25 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id WAA01101; Sun, 03 Jun 2012 22:35:16 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1SbGa4-000Mor-3y; Sun, 03 Jun 2012 22:35:16 +0300 Message-ID: <4FCBBC72.8070209@FreeBSD.org> Date: Sun, 03 Jun 2012 22:35:14 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:12.0) Gecko/20120503 Thunderbird/12.0.1 MIME-Version: 1.0 To: Attilio Rao References: <20120603.002554.119853142.iwasaki@jp.FreeBSD.org> <4FCB0FE5.4050607@FreeBSD.org> In-Reply-To: X-Enigmail-Version: 1.5pre Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: freebsd-acpi@FreeBSD.org, freebsd-arch@FreeBSD.org Subject: Re: cpu stopping [Was: preparation for x86/acpica/acpi_wakeup.c] X-BeenThere: freebsd-acpi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ACPI and power management development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Jun 2012 19:35:26 -0000 on 03/06/2012 12:54 Attilio Rao said the following: > 2012/6/3 Andriy Gapon : >> on 03/06/2012 00:39 Attilio Rao said the following: >>> The first thing to consider is that right now we only have 2 states >>> for CPUs: started and stopped. These states are controlled by >>> started_cpus and stopped_cpus masks respectively. It seems you really >>> want to add an intermediate level among the 2 where you have: started >>> -> suspended -> started -> suspended ... -> stopped and you need to >>> expand the mechanism for dealing with started and stopped cpus to do >>> that. I'm pretty sure this will be very helpful also for other >>> architectures that want to do the same. >> >> As the first thing I must admit that I haven't looked at the patch :-) >> >> >> But really I don't see why we need to differentiate between stopped and >> suspended state as both of them ultimately mean exactly the same thing - CPUs >> are spinning on some condition (and they are in a well-defined place and state). > > This is debeatable and I'm not sure I agree. > At some point we may want to implement CPU on-the-fly suspension for > CPUs which is a different event than "stopping" (where stopping will > be "permanent stopping" and suspending will be "possible to recover > suspension"). Right, but that should operate on the level above the current code. I.e. first stop all slave CPUs, than set state of a target CPU (which includes global view of that state), then resume all other CPUs. > The important thing about this is that we need to expand our model in > a way that it makes simple to add more states to the CPUs than simple > started/stopped. Right now we don't have any architecture for this in > place. I can't disagree with this, but I think that the current IPI-to-stop code is not a place for that. It's too low level. >> My view of how this should work is: >> - there can be only one master CPU that controls all other (slave) CPUs >> - the master sets entry and exit hooks >> - the master signals slaves to enter the stop state >> - the slaves execute the enter hook and start spinning on the release condition >> - the master does whatever it wants to do in this special system state >> - the master signals the slaves to resume >> - the slave exit the spin loop and execute the exit hook >> >> We have almost all of this in place. Only now we have different IPIs and >> different IPI handlers to do the job (cpustop_handler and cpususpend_handler). >> I think that the hooks model should be more universal. > > For hook you mean like a rendezvous handler? I'm not sure I understand > otherwise. Maybe, perhaps. I meant just a couple of function pointers. cpustop_restartfunc seems to be a better analogy. >> In my opinion, what really would deserve a completely independent path is the >> hard-stop case. As this can be invoked nested to the other cases. E.g. exotic >> situations like a breakpoint or a trap or a panic in the suspend or the normal >> stop code paths. > > What I'm really interested is expanding our model in a way that it can > handle multiple CPU states. Then it is just a matter of adding the > right states and it is all trivial work. > > And however, as already mentioned, I'm not sure I would assimilate > suspended = stopped. -- Andriy Gapon From owner-freebsd-acpi@FreeBSD.ORG Sun Jun 3 19:45:41 2012 Return-Path: Delivered-To: freebsd-acpi@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 24093106564A; Sun, 3 Jun 2012 19:45:41 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id DBFA78FC1F; Sun, 3 Jun 2012 19:45:39 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id WAA01145; Sun, 03 Jun 2012 22:45:34 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1SbGk1-000MpY-T6; Sun, 03 Jun 2012 22:45:33 +0300 Message-ID: <4FCBBEDD.5000604@FreeBSD.org> Date: Sun, 03 Jun 2012 22:45:33 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:12.0) Gecko/20120503 Thunderbird/12.0.1 MIME-Version: 1.0 To: Mitsuru IWASAKI References: <20120603.002554.119853142.iwasaki@jp.FreeBSD.org> <4FCB0FE5.4050607@FreeBSD.org> <20120603.234243.28389486.iwasaki@jp.FreeBSD.org> In-Reply-To: <20120603.234243.28389486.iwasaki@jp.FreeBSD.org> X-Enigmail-Version: 1.5pre Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: attilio@FreeBSD.org, freebsd-acpi@FreeBSD.org, freebsd-arch@FreeBSD.org Subject: Re: cpu stopping X-BeenThere: freebsd-acpi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ACPI and power management development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Jun 2012 19:45:41 -0000 on 03/06/2012 17:42 Mitsuru IWASAKI said the following: > Hi, thanks for comments. > >> As the first thing I must admit that I haven't looked at the patch :-) > > Never mind :) What I'm trying to do in the patches is just to unify > amd64/i386 independent part (acpi_wakeup.c) for the code maintenance, > so please let's commit it first, then start re-design the > cpususpend_handler(). In no way I am trying to delay your work :) Just shared my view on the design of cpu stopping code. >> But really I don't see why we need to differentiate between stopped and >> suspended state as both of them ultimately mean exactly the same thing - CPUs >> are spinning on some condition (and they are in a well-defined place and state). > > Yes, amd64/i386 cpususpend_handler() is very similar to cpustop_handler() > actually, some resume related procedures are added for suspend. > >> My view of how this should work is: >> - there can be only one master CPU that controls all other (slave) CPUs >> - the master sets entry and exit hooks > > Entry hook for suspending might be > ---- > ctx_fpusave(suspfpusave[cpu]); > wbinvd(); > CPU_SET_ATOMIC(cpu, &stopped_cpus); > ---- > > and for stopping is > ---- > /* Indicate that we are stopped */ > CPU_SET_ATOMIC(cpu, &stopped_cpus); > ---- > > Correct? Yes. The only nit is that CPU_SET_ATOMIC(cpu, &stopped_cpus) could be part of the wait loop prologue. No need to duplicate it in each hook. > I think stopping hook can be replaced with suspending hook. Perhaps... But let's not go into this topic just yet. > Exit hook for suspending is > ---- > pmap_init_pat(); > load_cr3(susppcbs[cpu]->pcb_cr3); > initializecpu(); > PCPU_SET(switchtime, 0); > PCPU_SET(switchticks, ticks); > [snip] > /* Resume MCA and local APIC */ > mca_resume(); > lapic_setup(0); > ---- > > For stopping should be > ---- > if (cpu == 0 && cpustop_restartfunc != NULL) { > cpustop_restartfunc(); > cpustop_restartfunc = NULL; > } > ---- > >> - the master signals slaves to enter the stop state >> - the slaves execute the enter hook and start spinning on the release condition >> - the master does whatever it wants to do in this special system state >> - the master signals the slaves to resume >> - the slave exit the spin loop and execute the exit hook > > I think it would be possible. However I personally think that > priority of x86/x86/mp_machdep.c is higher and more effective than > merging cpususpend/stop_handler(). I do not disagree. -- Andriy Gapon From owner-freebsd-acpi@FreeBSD.ORG Sun Jun 3 20:02:10 2012 Return-Path: Delivered-To: freebsd-acpi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9FBBC1065745; Sun, 3 Jun 2012 20:02:10 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-lpp01m010-f54.google.com (mail-lpp01m010-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id A3F3A8FC1A; Sun, 3 Jun 2012 20:02:09 +0000 (UTC) Received: by laai10 with SMTP id i10so3262639laa.13 for ; Sun, 03 Jun 2012 13:02:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=dUPpHTdTJTdWNZBt33McuZiSIzerq38ci95MZLizuIg=; b=ysizMijAvLhD1tY3B6qSu1V1CfNTU92DMr4EGBk+nYjYr+s9XZKnUc3M17gT2pzYJp O9KLbi3YI5KIdPtoD7CEgU9c2/kHdPLM8qpSL/UWydFcoePPSjW14l87IOZUO1Xypl6V te7tQYsfeztiGf8RKcONNtnAYOUw0o0A2rt/NuwapqjqQuJmXDOtCx7Qon4NUPcsyE5t mWRiiO0b6sCMu6jHeJuh56ipp6ki4HPslwVWjGUxtYcGGxBCsBD3n/wUl4/Xm22kWWfu xUDIR2KWaNdHleZLuC2aGDAC3D5Dkxtoqlf9P8ffzGdi//Skmz5ROJ1LnDj5BL/Ej7fV rewA== MIME-Version: 1.0 Received: by 10.152.103.11 with SMTP id fs11mr9876014lab.23.1338753721887; Sun, 03 Jun 2012 13:02:01 -0700 (PDT) Sender: asmrookie@gmail.com Received: by 10.112.27.65 with HTTP; Sun, 3 Jun 2012 13:02:01 -0700 (PDT) In-Reply-To: <4FCBBC72.8070209@FreeBSD.org> References: <20120603.002554.119853142.iwasaki@jp.FreeBSD.org> <4FCB0FE5.4050607@FreeBSD.org> <4FCBBC72.8070209@FreeBSD.org> Date: Sun, 3 Jun 2012 21:02:01 +0100 X-Google-Sender-Auth: xSOxkHrJHCFZkowbHu9KLCDOs4E Message-ID: From: Attilio Rao To: Andriy Gapon Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-acpi@freebsd.org, freebsd-arch@freebsd.org Subject: Re: cpu stopping [Was: preparation for x86/acpica/acpi_wakeup.c] X-BeenThere: freebsd-acpi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ACPI and power management development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Jun 2012 20:02:10 -0000 2012/6/3 Andriy Gapon : > on 03/06/2012 12:54 Attilio Rao said the following: >> 2012/6/3 Andriy Gapon : >>> on 03/06/2012 00:39 Attilio Rao said the following: >>>> The first thing to consider is that right now we only have 2 states >>>> for CPUs: started and stopped. These states are controlled by >>>> started_cpus and stopped_cpus masks respectively. It seems you really >>>> want to add an intermediate level among the 2 where you have: started >>>> -> suspended -> started -> suspended ... -> stopped and you need to >>>> expand the mechanism for dealing with started and stopped cpus to do >>>> that. I'm pretty sure this will be very helpful also for other >>>> architectures that want to do the same. >>> >>> As the first thing I must admit that I haven't looked at the patch :-) >>> >>> >>> But really I don't see why we need to differentiate between stopped and >>> suspended state as both of them ultimately mean exactly the same thing = - CPUs >>> are spinning on some condition (and they are in a well-defined place an= d state). >> >> This is debeatable and I'm not sure I agree. >> At some point we may want to implement CPU on-the-fly suspension for >> CPUs which is a different event than "stopping" (where stopping will >> be "permanent stopping" and suspending will be "possible to recover >> suspension"). > > Right, but that should operate on the level above the current code. > I.e. first stop all slave CPUs, than set state of a target CPU (which inc= ludes > global view of that state), then resume all other CPUs. > >> The important thing about this is that we need to expand our model in >> a way that it makes simple to add more states to the CPUs than simple >> started/stopped. Right now we don't have any architecture for this in >> place. > > I can't disagree with this, but I think that the current IPI-to-stop code= is not > a place for that. =C2=A0It's too low level. Yeah, I was referring in particular to the handling of the masks and few other things (stoppcbs, which could be rebased as suspendpcbs for that, etc.). The point I'm really trying to make is: our model is very very biased on the on/off case (started/stopped) and we need to abstract this and have a framework for adding several CPU states. After you have an abstracted model, you can simply make several states easi= lly. This is not a simple work and it is also less simple for synchronization, which right now is very much simplified/unhandled. I would be very happy if you or Mitsuru plan to work on that. Attilio --=20 Peace can only be achieved by understanding - A. Einstein From owner-freebsd-acpi@FreeBSD.ORG Mon Jun 4 11:07:30 2012 Return-Path: Delivered-To: freebsd-acpi@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0870C1065670 for ; Mon, 4 Jun 2012 11:07:30 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id CD7188FC16 for ; Mon, 4 Jun 2012 11:07:29 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q54B7TwE017329 for ; Mon, 4 Jun 2012 11:07:29 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q54B7SEN017327 for freebsd-acpi@FreeBSD.org; Mon, 4 Jun 2012 11:07:28 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 4 Jun 2012 11:07:28 GMT Message-Id: <201206041107.q54B7SEN017327@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-acpi@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-acpi@FreeBSD.org X-BeenThere: freebsd-acpi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ACPI and power management development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Jun 2012 11:07:30 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/164329 acpi [acpi] hw.acpi.thermal.tz0.temperature shows strange v o kern/163268 acpi [acpi_hp] fix driver detach in absence of CMI o kern/162859 acpi [acpi] ACPI battery/acline monitoring partialy working o kern/161715 acpi [acpi] Dell E6520 doesn't resume after ACPI suspend o kern/161713 acpi [acpi] Suspend on Dell E6520 o kern/160838 acpi [acpi] ACPI Battery Monitor Non-Functional o kern/160419 acpi [acpi_thermal] acpi_thermal kernel thread high CPU usa o kern/158689 acpi [acpi] value of sysctl hw.acpi.thermal.polling_rate ne o kern/154955 acpi [acpi] Keyboard or ACPI doesn't work on Lenovo S10-3 o kern/152438 acpi [acpi]: patch to acpi_asus(4) to add extra sysctls for o kern/152098 acpi [acpi] Lenovo T61p does not resume o i386/146715 acpi [acpi] Suspend works, resume not on a HP Probook 4510s o kern/145306 acpi [acpi]: Can't change brightness on HP ProBook 4510s o i386/143798 acpi [acpi] shutdown problem with SiS K7S5A o kern/143420 acpi [acpi] ACPI issues with Toshiba o kern/142009 acpi [acpi] [panic] Panic in AcpiNsGetAttachedObject o kern/139088 acpi [acpi] ACPI Exception: AE_AML_INFINITE_LOOP error o amd64/138210 acpi [acpi] acer aspire 5536 ACPI problems (S3, brightness, o kern/137042 acpi [acpi] hp laptop's lcd not wakes up after suspend to r o i386/136008 acpi [acpi] Dell Vostro 1310 will not shutdown (Requires us o bin/135349 acpi [patch] teach acpidump(8) to disassemble arbitrary mem o kern/132602 acpi [acpi] ACPI Problem with Intel SS4200: System does not p kern/128634 acpi [patch] fix acpi_asus(4) in asus a6f laptop o bin/126162 acpi [acpi] ACPI autoload failed : loading required module o kern/123039 acpi [acpi] ACPI AML_BUFFER_LIMIT errors during boot a i386/122887 acpi [panic] [atkbdc] 7.0-RELEASE on IBM HS20 panics immed o kern/121504 acpi [patch] Correctly set hw.acpi.osname on certain machin s kern/112544 acpi [acpi] [patch] Add High Precision Event Timer Driver f o kern/105537 acpi [acpi] problems in acpi on HP Compaq nc6320 o kern/91594 acpi [acpi] FreeBSD > 5.4 w/ACPI fails to detect Intel Pro/ o kern/73823 acpi [request] acpi / power-on by timer support o kern/56024 acpi ACPI suspend drains battery while in S3 32 problems total. From owner-freebsd-acpi@FreeBSD.ORG Wed Jun 6 18:32:28 2012 Return-Path: Delivered-To: freebsd-acpi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D69E4106564A for ; Wed, 6 Jun 2012 18:32:28 +0000 (UTC) (envelope-from iwasaki@jp.FreeBSD.org) Received: from locore.org (ns01.locore.org [218.45.21.227]) by mx1.freebsd.org (Postfix) with ESMTP id 724918FC1F for ; Wed, 6 Jun 2012 18:32:28 +0000 (UTC) Received: from localhost (celeron.v4.locore.org [192.168.0.10]) by locore.org (8.14.5/8.14.5/iwasaki) with ESMTP/inet id q56IWLA2050300 for ; Thu, 7 Jun 2012 03:32:21 +0900 (JST) (envelope-from iwasaki@jp.FreeBSD.org) Date: Thu, 07 Jun 2012 03:32:16 +0900 (JST) Message-Id: <20120607.033216.124057047.iwasaki@jp.FreeBSD.org> To: freebsd-acpi@freebsd.org From: Mitsuru IWASAKI In-Reply-To: <20120603.235839.133292778.iwasaki@jp.FreeBSD.org> References: <20120603.152326.105529138.iwasaki@jp.FreeBSD.org> <20120603.235839.133292778.iwasaki@jp.FreeBSD.org> X-Mailer: Mew version 3.3 on Emacs 20.7 / Mule 4.0 (HANANOEN) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: Re: preparation for x86/acpica/acpi_wakeup.c X-BeenThere: freebsd-acpi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ACPI and power management development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jun 2012 18:32:28 -0000 Hi, http://people.freebsd.org/~iwasaki/acpi/x86-acpi_wakeup-20120606.diff I've noticed that SVN rev 210777 commit log, said 'savectx() is only used for panic dump (dumppcb) and kdb (stoppcbs)'. So suspendctx() in i386 version should be merged into savectx(). I hope this would be the last patches. Thanks! From owner-freebsd-acpi@FreeBSD.ORG Wed Jun 6 19:03:44 2012 Return-Path: Delivered-To: freebsd-acpi@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3B7FA106564A; Wed, 6 Jun 2012 19:03:44 +0000 (UTC) (envelope-from iwasaki@jp.FreeBSD.org) Received: from locore.org (ns01.locore.org [218.45.21.227]) by mx1.freebsd.org (Postfix) with ESMTP id DCF5F8FC0A; Wed, 6 Jun 2012 19:03:43 +0000 (UTC) Received: from localhost (celeron.v4.locore.org [192.168.0.10]) by locore.org (8.14.5/8.14.5/iwasaki) with ESMTP/inet id q56J3gH9050606; Thu, 7 Jun 2012 04:03:42 +0900 (JST) (envelope-from iwasaki@jp.FreeBSD.org) Date: Thu, 07 Jun 2012 04:03:42 +0900 (JST) Message-Id: <20120607.040342.73368798.iwasaki@jp.FreeBSD.org> To: avg@FreeBSD.org From: Mitsuru IWASAKI In-Reply-To: <4FCBBEDD.5000604@FreeBSD.org> References: <4FCB0FE5.4050607@FreeBSD.org> <20120603.234243.28389486.iwasaki@jp.FreeBSD.org> <4FCBBEDD.5000604@FreeBSD.org> X-Mailer: Mew version 3.3 on Emacs 20.7 / Mule 4.0 (HANANOEN) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: attilio@FreeBSD.org, freebsd-acpi@FreeBSD.org, freebsd-arch@FreeBSD.org Subject: Re: cpu stopping X-BeenThere: freebsd-acpi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ACPI and power management development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jun 2012 19:03:44 -0000 Hi, I've created the patches of experimental implementation based on discussion so far. http://people.freebsd.org/~iwasaki/acpi/cpustop_hook-20120606.diff In acpi_wakeup.c, cpususpend_handler() and susppcbs are replaced with cpustop_handler() and stoppcbs. This is for RELENG_9 and only for i386 but I think it's enough for the start. From: Andriy Gapon Subject: Re: cpu stopping Date: Sun, 03 Jun 2012 22:45:33 +0300 Message-ID: <4FCBBEDD.5000604@FreeBSD.org> > > Never mind :) What I'm trying to do in the patches is just to unify > > amd64/i386 independent part (acpi_wakeup.c) for the code maintenance, > > so please let's commit it first, then start re-design the > > cpususpend_handler(). > > In no way I am trying to delay your work :) > Just shared my view on the design of cpu stopping code. I got it :) > >> My view of how this should work is: > >> - there can be only one master CPU that controls all other (slave) CPUs > >> - the master sets entry and exit hooks > > > > Entry hook for suspending might be > > ---- > > ctx_fpusave(suspfpusave[cpu]); > > wbinvd(); > > CPU_SET_ATOMIC(cpu, &stopped_cpus); > > ---- > > > > and for stopping is > > ---- > > /* Indicate that we are stopped */ > > CPU_SET_ATOMIC(cpu, &stopped_cpus); > > ---- > > > > Correct? > > Yes. The only nit is that CPU_SET_ATOMIC(cpu, &stopped_cpus) could be part of > the wait loop prologue. No need to duplicate it in each hook. OK, I did so. I hope the patch is not far from your idea. Thanks! From owner-freebsd-acpi@FreeBSD.ORG Wed Jun 6 20:16:20 2012 Return-Path: Delivered-To: freebsd-acpi@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 01E35106567C; Wed, 6 Jun 2012 20:16:20 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id D79468FC1C; Wed, 6 Jun 2012 20:16:18 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id XAA09341; Wed, 06 Jun 2012 23:16:11 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1ScMeJ-0009IJ-3l; Wed, 06 Jun 2012 23:16:11 +0300 Message-ID: <4FCFBA89.9030105@FreeBSD.org> Date: Wed, 06 Jun 2012 23:16:09 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:12.0) Gecko/20120503 Thunderbird/12.0.1 MIME-Version: 1.0 To: Mitsuru IWASAKI References: <4FCB0FE5.4050607@FreeBSD.org> <20120603.234243.28389486.iwasaki@jp.FreeBSD.org> <4FCBBEDD.5000604@FreeBSD.org> <20120607.040342.73368798.iwasaki@jp.FreeBSD.org> In-Reply-To: <20120607.040342.73368798.iwasaki@jp.FreeBSD.org> X-Enigmail-Version: 1.5pre Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: attilio@FreeBSD.org, freebsd-acpi@FreeBSD.org, freebsd-arch@FreeBSD.org Subject: Re: cpu stopping X-BeenThere: freebsd-acpi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ACPI and power management development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jun 2012 20:16:20 -0000 on 06/06/2012 22:03 Mitsuru IWASAKI said the following: > Hi, > > I've created the patches of experimental implementation based on > discussion so far. > > http://people.freebsd.org/~iwasaki/acpi/cpustop_hook-20120606.diff > > In acpi_wakeup.c, cpususpend_handler() and susppcbs are replaced with > cpustop_handler() and stoppcbs. > > This is for RELENG_9 and only for i386 but I think it's enough for the > start. I think that there is no need for DPCPU here. All (affected) CPUs should see the same hook, IMO. At least I can not imagine the case where something else would be required. Also, it might make sense to provide a void pointer as a potential context for for the context. As Attilio has said before this has many similarities to what smp_rendezvous does, just for different kind of situations. > From: Andriy Gapon > Subject: Re: cpu stopping > Date: Sun, 03 Jun 2012 22:45:33 +0300 > Message-ID: <4FCBBEDD.5000604@FreeBSD.org> > >>> Never mind :) What I'm trying to do in the patches is just to unify >>> amd64/i386 independent part (acpi_wakeup.c) for the code maintenance, >>> so please let's commit it first, then start re-design the >>> cpususpend_handler(). >> >> In no way I am trying to delay your work :) >> Just shared my view on the design of cpu stopping code. > > I got it :) > >>>> My view of how this should work is: >>>> - there can be only one master CPU that controls all other (slave) CPUs >>>> - the master sets entry and exit hooks >>> >>> Entry hook for suspending might be >>> ---- >>> ctx_fpusave(suspfpusave[cpu]); >>> wbinvd(); >>> CPU_SET_ATOMIC(cpu, &stopped_cpus); >>> ---- >>> >>> and for stopping is >>> ---- >>> /* Indicate that we are stopped */ >>> CPU_SET_ATOMIC(cpu, &stopped_cpus); >>> ---- >>> >>> Correct? >> >> Yes. The only nit is that CPU_SET_ATOMIC(cpu, &stopped_cpus) could be part of >> the wait loop prologue. No need to duplicate it in each hook. > > OK, I did so. > > I hope the patch is not far from your idea. > > Thanks! -- Andriy Gapon From owner-freebsd-acpi@FreeBSD.ORG Thu Jun 7 08:10:37 2012 Return-Path: Delivered-To: freebsd-acpi@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6B929106564A; Thu, 7 Jun 2012 08:10:37 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 5BE418FC16; Thu, 7 Jun 2012 08:10:35 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id LAA16671; Thu, 07 Jun 2012 11:10:33 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1ScXnd-000CXk-Aq; Thu, 07 Jun 2012 11:10:33 +0300 Message-ID: <4FD061F8.6030905@FreeBSD.org> Date: Thu, 07 Jun 2012 11:10:32 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:12.0) Gecko/20120503 Thunderbird/12.0.1 MIME-Version: 1.0 To: Jung-uk Kim References: <1337319129.2915.4.camel@powernoodle-l7> <4FB6765A.2050307@FreeBSD.org> <1337710214.2916.8.camel@powernoodle-l7.corp.yahoo.com> <20120525163653.b61a08e2.lists@yamagi.org> <4FBFA9A9.7020806@FreeBSD.org> <4FBFBD39.7000105@FreeBSD.org> <4FBFDFFB.9020501@FreeBSD.org> <4FBFE624.1020208@FreeBSD.org> <20120526090233.f638c1d2.lists@yamagi.org> <4FC0A3A1.80200@FreeBSD.org> <4FC7D464.20602@FreeBSD.org> <4FCFD2A1.60706@FreeBSD.org> <4FCFE178.9080505@FreeBSD.org> In-Reply-To: <4FCFE178.9080505@FreeBSD.org> X-Enigmail-Version: 1.5pre Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: sbruno@FreeBSD.org, freebsd-acpi@FreeBSD.org, freebsd-stable@FreeBSD.org, Yamagi Burmeister , seanbru@yahoo-inc.com Subject: Re: [stable 9] broken hwpstate calls X-BeenThere: freebsd-acpi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ACPI and power management development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Jun 2012 08:10:37 -0000 on 07/06/2012 02:02 Jung-uk Kim said the following: > Any way, hwpstate still isn't quite right even without your patch. > > sys/kern/kern_cpu.c cpufreq_curr_sysctl() -> CPUFREQ_SET() -> /* for all > CPU devices */ cf_set_method() -> /* thread_lock(), sched_bind(), ... */ > CPUFREQ_DRV_SET() -> sys/x86/cpufreq/hwpstate.c hwpstate_set() -> > hwpstate_goto_pstate() /* for each CPU unit */ /* thread_lock(), > sched_bind(), ... */ Oh, I didn't realize that there was the cpufreq-level loop over all CPUs! That really sucks. Maybe some day we should accept that different CPUs could legitimately be in different P-states and provide support for that throughout the stack (from powerd to drivers). -- Andriy Gapon From owner-freebsd-acpi@FreeBSD.ORG Thu Jun 7 08:38:35 2012 Return-Path: Delivered-To: freebsd-acpi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BB485106564A; Thu, 7 Jun 2012 08:38:35 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-bk0-f54.google.com (mail-bk0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 86DE88FC0C; Thu, 7 Jun 2012 08:38:34 +0000 (UTC) Received: by bkvi18 with SMTP id i18so348139bkv.13 for ; Thu, 07 Jun 2012 01:38:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=ERyFbkuNwCmzNOKV3QtMfkEBmQAQj3eOzYZNCA2wypk=; b=pV6umtVrQvV21vAdsjgO2HnitkIK3kN5m4FgJMRZgE3PxL04mqxTXUxsLVBybhq3KE uYjJw+LSnW4F+pXO/+2dVSUV/x3VOpFPoOeghqdmGMpAj8XKCQiuGwZzfks94WqrJ6IQ 9BLgrISIbjIt27K0QRQweJn+ctWcKupUIMKFIWW5kFf220OWKIk+aQOKqzHjoHRRvMjq YJRYSnXnILsWkKhZ+s0w5+vFt/P/7QkxI1H4mdOwu1rmGrHwkdBXVu8jJiA577V7KJjQ jv19wXGctyz9PLYvFYlot/oLj+cYmK9VdOhuJVOEe0ibdm3dMq8lcWVLHj3WYpXJipHS hzoA== Received: by 10.205.137.5 with SMTP id im5mr768348bkc.45.1339058313299; Thu, 07 Jun 2012 01:38:33 -0700 (PDT) Received: from mavbook2.mavhome.dp.ua (pc.mavhome.dp.ua. [212.86.226.226]) by mx.google.com with ESMTPS id n19sm2602990bkv.14.2012.06.07.01.38.30 (version=SSLv3 cipher=OTHER); Thu, 07 Jun 2012 01:38:32 -0700 (PDT) Sender: Alexander Motin Message-ID: <4FD06885.4050507@FreeBSD.org> Date: Thu, 07 Jun 2012 11:38:29 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:12.0) Gecko/20120506 Thunderbird/12.0.1 MIME-Version: 1.0 To: Andriy Gapon References: <1337319129.2915.4.camel@powernoodle-l7> <4FB6765A.2050307@FreeBSD.org> <1337710214.2916.8.camel@powernoodle-l7.corp.yahoo.com> <20120525163653.b61a08e2.lists@yamagi.org> <4FBFA9A9.7020806@FreeBSD.org> <4FBFBD39.7000105@FreeBSD.org> <4FBFDFFB.9020501@FreeBSD.org> <4FBFE624.1020208@FreeBSD.org> <20120526090233.f638c1d2.lists@yamagi.org> <4FC0A3A1.80200@FreeBSD.org> <4FC7D464.20602@FreeBSD.org> <4FCFD2A1.60706@FreeBSD.org> <4FCFE178.9080505@FreeBSD.org> <4FD061F8.6030905@FreeBSD.org> In-Reply-To: <4FD061F8.6030905@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-stable@FreeBSD.org, freebsd-acpi@FreeBSD.org, sbruno@FreeBSD.org, Yamagi Burmeister , seanbru@yahoo-inc.com, Jung-uk Kim Subject: Re: [stable 9] broken hwpstate calls X-BeenThere: freebsd-acpi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ACPI and power management development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Jun 2012 08:38:35 -0000 On 06/07/12 11:10, Andriy Gapon wrote: > on 07/06/2012 02:02 Jung-uk Kim said the following: >> Any way, hwpstate still isn't quite right even without your patch. >> >> sys/kern/kern_cpu.c cpufreq_curr_sysctl() -> CPUFREQ_SET() -> /* for all >> CPU devices */ cf_set_method() -> /* thread_lock(), sched_bind(), ... */ >> CPUFREQ_DRV_SET() -> sys/x86/cpufreq/hwpstate.c hwpstate_set() -> >> hwpstate_goto_pstate() /* for each CPU unit */ /* thread_lock(), >> sched_bind(), ... */ > > Oh, I didn't realize that there was the cpufreq-level loop over all CPUs! > That really sucks. > > Maybe some day we should accept that different CPUs could legitimately be in > different P-states and provide support for that throughout the stack (from > powerd to drivers). Support for different P-states on different CPUs can be useful if CPUs have different capabilities. I believe it is very rare, but possible. At this moment cpufreq should set for each CPU frequency closest to one that was set on BSP. It should be possible to make powerd to read sets of frequencies from all CPUs and do the same, just more intelligently. Same time using very different frequencies for different CPUs can IMHO be very problematic even in theory. For SMP systems it is quite difficult (because of threads migration and possible inter-operations of multiple threads) to identify cases when even global frequency can be reduced without proportional performance penalty. Making in per-CPU multiplies number of options and requires awareness from the scheduler. -- Alexander Motin From owner-freebsd-acpi@FreeBSD.ORG Thu Jun 7 18:05:18 2012 Return-Path: Delivered-To: freebsd-acpi@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A7121106564A; Thu, 7 Jun 2012 18:05:18 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id BA8D28FC15; Thu, 7 Jun 2012 18:04:54 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id VAA22855; Thu, 07 Jun 2012 21:04:53 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1Sch4n-000DE2-6R; Thu, 07 Jun 2012 21:04:53 +0300 Message-ID: <4FD0ED44.9040402@FreeBSD.org> Date: Thu, 07 Jun 2012 21:04:52 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:12.0) Gecko/20120503 Thunderbird/12.0.1 MIME-Version: 1.0 To: Alexander Motin References: <1337319129.2915.4.camel@powernoodle-l7> <4FB6765A.2050307@FreeBSD.org> <1337710214.2916.8.camel@powernoodle-l7.corp.yahoo.com> <20120525163653.b61a08e2.lists@yamagi.org> <4FBFA9A9.7020806@FreeBSD.org> <4FBFBD39.7000105@FreeBSD.org> <4FBFDFFB.9020501@FreeBSD.org> <4FBFE624.1020208@FreeBSD.org> <20120526090233.f638c1d2.lists@yamagi.org> <4FC0A3A1.80200@FreeBSD.org> <4FC7D464.20602@FreeBSD.org> <4FCFD2A1.60706@FreeBSD.org> <4FCFE178.9080505@FreeBSD.org> <4FD061F8.6030905@FreeBSD.org> <4FD06885.4050507@FreeBSD.org> In-Reply-To: <4FD06885.4050507@FreeBSD.org> X-Enigmail-Version: 1.5pre Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-acpi@FreeBSD.org Subject: Re: [stable 9] broken hwpstate calls X-BeenThere: freebsd-acpi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ACPI and power management development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Jun 2012 18:05:18 -0000 on 07/06/2012 11:38 Alexander Motin said the following: > On 06/07/12 11:10, Andriy Gapon wrote: >> on 07/06/2012 02:02 Jung-uk Kim said the following: >>> Any way, hwpstate still isn't quite right even without your patch. >>> >>> sys/kern/kern_cpu.c cpufreq_curr_sysctl() -> CPUFREQ_SET() -> /* for all >>> CPU devices */ cf_set_method() -> /* thread_lock(), sched_bind(), ... */ >>> CPUFREQ_DRV_SET() -> sys/x86/cpufreq/hwpstate.c hwpstate_set() -> >>> hwpstate_goto_pstate() /* for each CPU unit */ /* thread_lock(), >>> sched_bind(), ... */ >> >> Oh, I didn't realize that there was the cpufreq-level loop over all CPUs! >> That really sucks. >> >> Maybe some day we should accept that different CPUs could legitimately be in >> different P-states and provide support for that throughout the stack (from >> powerd to drivers). > > Support for different P-states on different CPUs can be useful if CPUs have > different capabilities. Not sure what you mean... I was talking about setting different CPUs to different P-states based on the per-CPU conditions (e.g. utilization). I certainly didn't mean to talk about heterogeneous P-state definitions or any other heterogeneous silicon issues. > I believe it is very rare, but possible. At this moment > cpufreq should set for each CPU frequency closest to one that was set on BSP. It > should be possible to make powerd to read sets of frequencies from all CPUs and > do the same, just more intelligently. > > Same time using very different frequencies for different CPUs can IMHO be very > problematic even in theory. For SMP systems it is quite difficult (because of > threads migration and possible inter-operations of multiple threads) to identify > cases when even global frequency can be reduced without proportional performance > penalty. Making in per-CPU multiplies number of options and requires awareness > from the scheduler. I humbly disagree. I think that it's not a job of scheduler to be overly smart when power-saving policies are in effect. IMO, scheduler should just do its own job and powerd should react to individual loads of CPUs. Where latencies really matter there powerd should not be used (or perhaps used with some different policy skewed towards performance vs economy). Also, Linux does it, so it must at least doable :-) -- Andriy Gapon From owner-freebsd-acpi@FreeBSD.ORG Thu Jun 7 19:13:40 2012 Return-Path: Delivered-To: freebsd-acpi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9C1AC1065672; Thu, 7 Jun 2012 19:13:40 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-bk0-f54.google.com (mail-bk0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id DF3B88FC16; Thu, 7 Jun 2012 19:13:39 +0000 (UTC) Received: by bkvi18 with SMTP id i18so1112809bkv.13 for ; Thu, 07 Jun 2012 12:13:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=4Mo4EtGGVdJ+jNcjH516Nz0H40M7CmU8Rv4dbLPzYgU=; b=ji/k5fvVbEN+/UXFvZGFYb9VNUfGDMJ3I0NOv1U3lho+iVQpa0w/afpwXH/cKkLYhe MjtoYRBgit8Sqfc/fe1u+L6VWPs2tB6x+SrfMXtQkaIJ5f9dFPbvvg+3o7BQspQwu/7F Q/4PzDQF2xeGN//PUdoAI7jTlBVAl6bTzaHEYIFtvkMeY01F76W6g+DKY7e4r7U7101O uGzIiuW9zMnXm79B3Xr9B4oKa5lq7+m+K3eukTmzHn4bDocL8hnDTnkfvD+mOspFEmre JU3HEOhCzVFEPQEKkKE+qyKNsI6FDVzqcxUkH0AP5fxY7uIPQifyXZxvk7svHIbZY/uh j/SQ== Received: by 10.204.154.144 with SMTP id o16mr2658597bkw.107.1339096412697; Thu, 07 Jun 2012 12:13:32 -0700 (PDT) Received: from mavbook2.mavhome.dp.ua (pc.mavhome.dp.ua. [212.86.226.226]) by mx.google.com with ESMTPS id e20sm5022892bkw.3.2012.06.07.12.13.31 (version=SSLv3 cipher=OTHER); Thu, 07 Jun 2012 12:13:31 -0700 (PDT) Sender: Alexander Motin Message-ID: <4FD0FD5A.9070807@FreeBSD.org> Date: Thu, 07 Jun 2012 22:13:30 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:12.0) Gecko/20120506 Thunderbird/12.0.1 MIME-Version: 1.0 To: Andriy Gapon References: <1337319129.2915.4.camel@powernoodle-l7> <4FB6765A.2050307@FreeBSD.org> <1337710214.2916.8.camel@powernoodle-l7.corp.yahoo.com> <20120525163653.b61a08e2.lists@yamagi.org> <4FBFA9A9.7020806@FreeBSD.org> <4FBFBD39.7000105@FreeBSD.org> <4FBFDFFB.9020501@FreeBSD.org> <4FBFE624.1020208@FreeBSD.org> <20120526090233.f638c1d2.lists@yamagi.org> <4FC0A3A1.80200@FreeBSD.org> <4FC7D464.20602@FreeBSD.org> <4FCFD2A1.60706@FreeBSD.org> <4FCFE178.9080505@FreeBSD.org> <4FD061F8.6030905@FreeBSD.org> <4FD06885.4050507@FreeBSD.org> <4FD0ED44.9040402@FreeBSD.org> In-Reply-To: <4FD0ED44.9040402@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-acpi@FreeBSD.org Subject: Re: [stable 9] broken hwpstate calls X-BeenThere: freebsd-acpi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ACPI and power management development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Jun 2012 19:13:40 -0000 On 06/07/12 21:04, Andriy Gapon wrote: > on 07/06/2012 11:38 Alexander Motin said the following: >> On 06/07/12 11:10, Andriy Gapon wrote: >>> on 07/06/2012 02:02 Jung-uk Kim said the following: >>>> Any way, hwpstate still isn't quite right even without your patch. >>>> >>>> sys/kern/kern_cpu.c cpufreq_curr_sysctl() -> CPUFREQ_SET() -> /* for all >>>> CPU devices */ cf_set_method() -> /* thread_lock(), sched_bind(), ... */ >>>> CPUFREQ_DRV_SET() -> sys/x86/cpufreq/hwpstate.c hwpstate_set() -> >>>> hwpstate_goto_pstate() /* for each CPU unit */ /* thread_lock(), >>>> sched_bind(), ... */ >>> >>> Oh, I didn't realize that there was the cpufreq-level loop over all CPUs! >>> That really sucks. >>> >>> Maybe some day we should accept that different CPUs could legitimately be in >>> different P-states and provide support for that throughout the stack (from >>> powerd to drivers). >> >> Support for different P-states on different CPUs can be useful if CPUs have >> different capabilities. > > Not sure what you mean... I was talking about setting different CPUs to > different P-states based on the per-CPU conditions (e.g. utilization). I > certainly didn't mean to talk about heterogeneous P-state definitions or any > other heterogeneous silicon issues. As you wish, but at this moment it is the only realistic application I see. As I've told below, setting different frequencies to different cores without scheduler awareness is a bad idea. >> I believe it is very rare, but possible. At this moment >> cpufreq should set for each CPU frequency closest to one that was set on BSP. It >> should be possible to make powerd to read sets of frequencies from all CPUs and >> do the same, just more intelligently. >> >> Same time using very different frequencies for different CPUs can IMHO be very >> problematic even in theory. For SMP systems it is quite difficult (because of >> threads migration and possible inter-operations of multiple threads) to identify >> cases when even global frequency can be reduced without proportional performance >> penalty. Making in per-CPU multiplies number of options and requires awareness >> from the scheduler. > > I humbly disagree. I think that it's not a job of scheduler to be overly smart > when power-saving policies are in effect. IMO, scheduler should just do its own > job and powerd should react to individual loads of CPUs. Where latencies really > matter there powerd should not be used (or perhaps used with some different > policy skewed towards performance vs economy). Scheduler usually operates in terms of milliseconds or less. powerd operates in best case in terms of fractions of seconds (or it will eat more power then save). Unless you are doing some heavy CPU-bound math without any context switches, it won't work well without scheduler aware about available computation resources. > Also, Linux does it, so it must at least doable :-) I don't know whether or how Linux does it. If you know how to do it effectively -- welcome, be my guest. :) -- Alexander Motin From owner-freebsd-acpi@FreeBSD.ORG Fri Jun 8 01:10:16 2012 Return-Path: Delivered-To: freebsd-acpi@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DA0D81065670 for ; Fri, 8 Jun 2012 01:10:16 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id AE45D8FC12 for ; Fri, 8 Jun 2012 01:10:16 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q581AGTL055505 for ; Fri, 8 Jun 2012 01:10:16 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q581AGJ2055504; Fri, 8 Jun 2012 01:10:16 GMT (envelope-from gnats) Date: Fri, 8 Jun 2012 01:10:16 GMT Message-Id: <201206080110.q581AGJ2055504@freefall.freebsd.org> To: freebsd-acpi@FreeBSD.org From: bazzoola Cc: Subject: Re: kern/105537: [acpi] problems in acpi on HP Compaq nc6320 X-BeenThere: freebsd-acpi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: bazzoola List-Id: ACPI and power management development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jun 2012 01:10:16 -0000 The following reply was made to PR kern/105537; it has been noted by GNATS. From: bazzoola To: bug-followup@FreeBSD.org, emin@mccme.ru Cc: Subject: Re: kern/105537: [acpi] problems in acpi on HP Compaq nc6320 Date: Thu, 7 Jun 2012 21:08:33 -0400 This is still an issue on HP 530 Laptop. I am willing to debug it. Just = point me where to start.