From owner-freebsd-mips@FreeBSD.ORG Fri Jan 29 06:42:39 2010 Return-Path: Delivered-To: freebsd-mips@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7F7AE106566B for ; Fri, 29 Jan 2010 06:42:39 +0000 (UTC) (envelope-from juli@clockworksquid.com) Received: from mail-px0-f183.google.com (mail-px0-f183.google.com [209.85.216.183]) by mx1.freebsd.org (Postfix) with ESMTP id 5CE5B8FC12 for ; Fri, 29 Jan 2010 06:42:39 +0000 (UTC) Received: by pxi13 with SMTP id 13so1174302pxi.3 for ; Thu, 28 Jan 2010 22:42:39 -0800 (PST) MIME-Version: 1.0 Sender: juli@clockworksquid.com Received: by 10.142.3.25 with SMTP id 25mr312416wfc.200.1264747359071; Thu, 28 Jan 2010 22:42:39 -0800 (PST) In-Reply-To: <85D9D383-29A3-4F09-A2FE-61E4EA85CE9B@lakerest.net> References: <20100128.132114.1004138037722505681.imp@bsdimp.com> <66207A08-F691-4603-A6C5-9C675414C91E@lakerest.net> <85D9D383-29A3-4F09-A2FE-61E4EA85CE9B@lakerest.net> From: Juli Mallett Date: Thu, 28 Jan 2010 22:42:19 -0800 X-Google-Sender-Auth: 0664fad28df8094f Message-ID: To: Randall Stewart Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Cc: freebsd-mips@freebsd.org, Neel Natu Subject: Re: Code review: groundwork for SMP X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Jan 2010 06:42:39 -0000 On Thu, Jan 28, 2010 at 21:28, Randall Stewart wrote: >> [ Using a single wired TLB entry for kstack and pcpu ] > > Which means you have a big array that you are offsetting. Not really =97 you can have a structure at 0xc000000000000000u (or the same >> 32) with two pointers in it, even, one to pcpu and one to KSTACK_PAGES direct-mapped, contiguous pages. Then you can load the kstack address or the pcpu base very quickly. Of course, you can even have a single wired entry consisting of the pcpu data and then put a pointer to the top of the kstack in it. I don't think you can get by with no wired TLB entries, but you also don't have to index a big array. The nice thing about setting up a per-CPU TLB entry (you have to set up at least one, the kstack, in order to be able to handle exceptions) is that then you need only access offsets into it that are known at compile time and constant no matter what CPU you're running on. Load the kstack by doing "ld sp, 0(0xc...)" and load the pcpu address by doing "ld t0, 8(0xc....)". Two wired entries lets you get rid of the indirection, but you can get by with one and still not have to do (1) run-time computation of the index into some array (2) possibly very expensive getting of the cpuid. > I was even thinking get a LARGE entry.. one that is say 8 Meg > for the kernel.. covering all text/data etc... with this > new super page stuff. of course I have never looked into how > its implemented.. That would be easy to do, but what would be the benefits of accessing that data through a wired TLB entry instead of the direct map? > Yes, you pay an index reference for every access .. or at > least one to setup a pointer.. but I think that it much cheaper > than a TLB miss is... (words for imp to think about)... Yes, TLB misses are very slow. Your desire to avoid adding another wired entry seems pretty reasonable. I think that using a single wired TLB entry for a mux or for both the kstack and pcpu is easy and usable. I feel like just wiring the kstack and putting a direct-mapped, sometimes-recomputed pointer to the pcpu into gp is the best combination in the long run =97 even just loading an immediate 64-bit address is pretty slow wrt how often things in the PCPU are accessed in SMP kernels. Juli.