From owner-freebsd-arch@FreeBSD.ORG  Fri Dec 17 06:28:47 2004
Return-Path: <owner-freebsd-arch@FreeBSD.ORG>
Delivered-To: freebsd-arch@freebsd.org
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 8A95816A4CF
	for <freebsd-arch@freebsd.org>; Fri, 17 Dec 2004 06:28:47 +0000 (GMT)
Received: from rproxy.gmail.com (rproxy.gmail.com [64.233.170.192])
	by mx1.FreeBSD.org (Postfix) with ESMTP id 1670543D31
	for <freebsd-arch@freebsd.org>; Fri, 17 Dec 2004 06:28:47 +0000 (GMT)
	(envelope-from joseph.koshy@gmail.com)
Received: by rproxy.gmail.com with SMTP id j1so1269268rnf
        for <freebsd-arch@freebsd.org>; Thu, 16 Dec 2004 22:28:46 -0800 (PST)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
	s=beta; d=gmail.com;
	h=received:message-id:date:from:reply-to:to:subject:mime-version:content-type:content-transfer-encoding;
	b=GDp1IowxgGwfxjvN+RDdA61bsGyzZ+zIHd6gRzUHlq+Xzqukc0U2+iUBdi+U+dr7wl/EQdsP81w6a6XIiFXvoW9flIykndH+QwLkv9O08OKAxEuj+j2kwX8UzMH9HgR5VJMP2KTxilrYcTIoLQdSA67bCf2+RSjZ+NURhAHB2n8=
Received: by 10.38.163.80 with SMTP id l80mr103407rne;
        Thu, 16 Dec 2004 22:28:46 -0800 (PST)
Received: by 10.38.209.11 with HTTP; Thu, 16 Dec 2004 22:28:46 -0800 (PST)
Message-ID: <84dead7204121622283c51d41a@mail.gmail.com>
Date: Fri, 17 Dec 2004 11:58:46 +0530
From: Joseph Koshy <joseph.koshy@gmail.com>
To: freebsd-arch@freebsd.org
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Subject: Eventhandlers for CPU state changes
X-BeenThere: freebsd-arch@freebsd.org
X-Mailman-Version: 2.1.1
Precedence: list
Reply-To: Joseph Koshy <joseph.koshy@gmail.com>
List-Id: Discussion related to FreeBSD architecture
	<freebsd-arch.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-arch>,
	<mailto:freebsd-arch-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-arch>
List-Post: <mailto:freebsd-arch@freebsd.org>
List-Help: <mailto:freebsd-arch-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-arch>,
	<mailto:freebsd-arch-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 17 Dec 2004 06:28:47 -0000

We currently allow CPUs to be enabled and disabled via the 
sysctl "machdep.hlt_cpus".  CPU enabling and disabling can
happen at any time in our current design.

It would be useful to have an eventhandler hook so that kernel
modules can get notified when a CPU is enabled or disabled.
For example, schedulers, memory allocators and drivers that
use MSRs inside the CPU could use this information.

Here is a straightforward attempt at implementing such an
eventhandler:
    http://perforce.freebsd.org/changeView.cgi?CH=67118

The question however, is about the right kind of semantics for
such an eventhandler  (The eventhandler above is at best
"advisory" in nature).

When we notify a module that "CPU X is going to go away", we
may still want to allow that module to run on that CPU to cleanup
some state (e.g., disable some MSRs).  However we may also wish
to prevent other kernel threads from getting scheduled onto the
to-be-disabled CPU inadvertently.

One way of getting around this problem is for schedulers to register
two eventhandlers.  The first with priority EVENTHANDLER_PRI_FIRST
marks CPU X as about to be disabled.  Threads will not then get scheduled
onto CPU X unless bound to it.  The second with priority 
EVENTHANDLER_PRI_LAST will actually disable CPU X.

Suggestions?  Comments?