From owner-freebsd-mips@FreeBSD.ORG Sat Jan 30 12:39:49 2010 Return-Path: Delivered-To: freebsd-mips@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 45A46106566C; Sat, 30 Jan 2010 12:39:49 +0000 (UTC) (envelope-from c.jayachandran@gmail.com) Received: from mail-pz0-f194.google.com (mail-pz0-f194.google.com [209.85.222.194]) by mx1.freebsd.org (Postfix) with ESMTP id 128048FC13; Sat, 30 Jan 2010 12:39:48 +0000 (UTC) Received: by pzk32 with SMTP id 32so92471pzk.27 for ; Sat, 30 Jan 2010 04:39:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=z+iao1q3Mpf3RIVXKDIiMOMB9I/tFoSTCZpmFoQdg3c=; b=oPFjJ0+Y6X6uj81wWgsd45NIjfVNUDgqRy1TNIoEBLMivB/dVroG8lL7512rDQ5f9s o3grdgY5HcUdAjHwxeR9HuG31gDgHcwdIs35vzAkOKAaPidJ7JKhBfY7qQIxjnX8RMbY rOkzeNbN6p9NI4PI627bx3/38I38jdlPL/lDk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=WtRwP3nEGDhcTACTV7qY4TG89iUcvMGONJKot2sR9mksF/VoTlzBo3ogvUtODvGhli uJiXM/BQWSizf2bMnS2apf6dkFDcYXuGcz3teAZSZ4Lw/dBg1+33s/lS41mf+9/cka3v QfGWA7gwBOtMbUG5K1fBNmAHOuYz3bxxwWpeA= MIME-Version: 1.0 Received: by 10.141.88.19 with SMTP id q19mr1449792rvl.89.1264855188540; Sat, 30 Jan 2010 04:39:48 -0800 (PST) In-Reply-To: <20100129.100052.1013538172663276257.imp@bsdimp.com> References: <85D9D383-29A3-4F09-A2FE-61E4EA85CE9B@lakerest.net> <20100129.100052.1013538172663276257.imp@bsdimp.com> Date: Sat, 30 Jan 2010 18:09:48 +0530 Message-ID: <98a59be81001300439o12ec3bf4pc5c03d4f1511297b@mail.gmail.com> From: "C. Jayachandran" To: "M. Warner Losh" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-mips@freebsd.org, neelnatu@gmail.com Subject: Re: Code review: groundwork for SMP X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Jan 2010 12:39:49 -0000 On Fri, Jan 29, 2010 at 10:30 PM, M. Warner Losh wrote: > Greetings one and all. =C2=A0Thanks for weighing in on this issue. > > In general, I agree with Neel here. =C2=A0But I also think we need to see > if we can be flexible and push this down into a per-cpu-type > decision (which differs slightly from a per-platform type because we > can have a CPU appearing in multiple platforms, or multiple CPUs > appearing within one platform). =C2=A0If we make it a per-cpu-type > solution, we could have a sys/mips/mips/pcpu_machdep.c which does the > normal SMP stuff, as well as having sys/mips/xlr/pcpu_machdep.c which > does something optimized for the XLR. =C2=A0Chances are good that differe= nt > CPUs will want to have different trade-offs here. =C2=A0We'd also need so= me > way to encode this in an include file, so there's some work to make > PCPU macro different for different CPUs... Yes, I think if the PCPU macros can be left to the specific implementation (even if that involves just ifdef in pcpu.h), we are fine. As I wrote earlier XLR preferred implementation would be to have the this in direct-mapped memory and have pointers in per-CPU scratch registers, and avoid TLB overhead. [...] > The XLR will have scheduler challenges as well. =C2=A0It will push the > design assumptions of ULE beyond the breaking point, I fear. > Hyperthreading already exists on intel, and ULE copes, a bit, with > it. =C2=A0But with the high number of threads each CPU can have, we may > need something with a little more smarts. =C2=A0Something that knows it > might be better to schedule two different processes on two different > cores, and leave some of the threads idle to reduce TLB pressure, for > example. Yes, this is one area we have not looked at much. > Per CPU scratch registers do not exist on MIPS, in general. =C2=A0Some CP= Us > have them, and many do not. =C2=A0CP0 registers are plentiful in more > modern designs, and some of them may even be useful for our needs. > However, mfc0 and mtc0 often have pipeline hazards associated with > them which will trip up the unwary. =C2=A0When reading the historical > errata for MIPS CPUs, we often find that this is where we need to do > the most workarounds. In XLR, there are 8 scratch registers, which can be accessed in a few cycles without any software visible hazards. So in our internal port we keep things like PCPU pointer and 'curthread' in scratch registers. > I guess this is a long way to say "I think we should commit Neel's > patches. =C2=A0We should work along two fronts: (1) implementing Juli's > idea of sharing kstack and pcpu data in one TLB and (2) making it so > that CPUs where this is sub-optimal can swap in their own > implementation." Another suggestion I have on Neel's implementation would be to allocate the pages (direct-mapped) at bootup, so that memory is not taken for cpus which are not available (XLR can have 4-32 cpus depending on the chip). Regards, JC.