From owner-freebsd-arch@FreeBSD.ORG Tue Jun 9 20:37:48 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B5DA41065670 for ; Tue, 9 Jun 2009 20:37:48 +0000 (UTC) (envelope-from marius@alchemy.franken.de) Received: from alchemy.franken.de (alchemy.franken.de [194.94.249.214]) by mx1.freebsd.org (Postfix) with ESMTP id 2CE0E8FC1E for ; Tue, 9 Jun 2009 20:37:47 +0000 (UTC) (envelope-from marius@alchemy.franken.de) Received: from alchemy.franken.de (localhost [127.0.0.1]) by alchemy.franken.de (8.14.3/8.14.3/ALCHEMY.FRANKEN.DE) with ESMTP id n59KBRoL051163; Tue, 9 Jun 2009 22:11:27 +0200 (CEST) (envelope-from marius@alchemy.franken.de) Received: (from marius@localhost) by alchemy.franken.de (8.14.3/8.14.3/Submit) id n59KBR2w051162; Tue, 9 Jun 2009 22:11:27 +0200 (CEST) (envelope-from marius) Date: Tue, 9 Jun 2009 22:11:27 +0200 From: Marius Strobl To: Jeff Roberson Message-ID: <20090609201127.GA50903@alchemy.franken.de> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: arch@freebsd.org Subject: Re: Dynamic pcpu, arm, mips, powerpc, sun, etc. help needed X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Jun 2009 20:37:49 -0000 On Wed, Jun 03, 2009 at 08:55:39PM -1000, Jeff Roberson wrote: > http://people.freebsd.org/~jeff/dpcpu.diff > > This patch implements dynamic per-cpu areas such that kernel code can do > the following in a header: > > DPCPU_DECLARE(uint64_t, foo); > > and this in source: > > DPCPU_DEFINE(uint64_t, foo) = 10; > > local = DPCPU_GET(foo); > DPCPU_SET(foo, 11); > > The dynamic per-cpu area of non-local cpus is accessable via > DPCPU_ID_{GET,SET,PTR}. > > If you provide an initializer as I used above that will be the default > value when all cpus come up. Otherwise it defaults to zero. This is > presently slightly more expensive than PCPU but much more flexible. > Things like id and curthread should stay in PCPU forever. > > I had to change the pcpu_init() call on every architecture to pass in > storage for the dynamic area. I didn't change the following three calls > because it wasn't immediately obvious how to allocate the memory: > > ./powerpc/booke/machdep.c: pcpu_init(pc, 0, sizeof(struct pcpu)); > ./mips/mips/machdep.c: pcpu_init(&__pcpu[0], 0, sizeof(struct pcpu)); > ./mips/mips/machdep.c: pcpu_init(pcpup, 0, sizeof(struct pcpu)); > > > I have not tested anything other than amd64. If you have a !amd64 > architecture, in particular any of the embedded architectures, I would > really appreciate it. Some of the arm boards postincrement the end > address to allocate early memory and some pre-decriment. Hopefully I got > it right. As for sparc64 allocating the storage for the dynamic area from end probably isn't a good idea as the pmap code assumes that the range from KERNBASE to end is covered by the pages allocated by and locked into the TLB for the kernel by the loader, so depending on the actual kernel size the DPCPU storage may be outside that region. The printf() will also blow at the location you moved it in sparc64_init() as the console isn't initialized at that point, yet. I think the best thing to do for the BSP would be to allocate its DPCPU storage via pmap_bootstrap_alloc() and use the direct mapping. This causes a bit of a chicken and egg problem though as MD parts of the per-CPU data are used to get pmap_bootstrap_alloc() working. Could you move the initialization of the DPCPU part to a dpcpu_init()? Marius