Date: Tue, 06 Aug 2002 19:00:33 -0700 From: Peter Wemm <peter@wemm.org> To: Terry Lambert <tlambert2@mindspring.com> Cc: Julian Elischer <julian@elischer.org>, Luigi Rizzo <rizzo@icir.org>, smp@freebsd.org Subject: Re: how to create per-cpu variables in SMP kernels ? Message-ID: <20020807020033.1F0842A7D6@canning.wemm.org> In-Reply-To: <3D504B0A.9FDB3A47@mindspring.com>
index | next in thread | previous in thread | raw e-mail
Terry Lambert wrote: > Julian Elischer wrote: > > On Tue, 6 Aug 2002, Terry Lambert wrote: > > > Luigi Rizzo wrote: > > > > > > NO. This can not work. > > > > > > The problem is that the per-CPU are is mapped into the same > > > location on each CPU -- and *totally inaccessible* to other CPUs. > > > > Terry, -current does not have multile page directories , one per cpu. > > (not any more). we use teh %fs register which is not used for anything > > else to have a special 'per-cpu segment'. The per-cpu mappings were > > being used at one stage but not any more... each pcpu area lives at a > > different virtual address now. > > > > basically, we do *all_pcpu[MAXCPU] where the index is achieved > > using the unused %fs register to make teh indexing work. > > > > in some places in the kernel we actually iterate through the elements. > > if they were not at different addreses this would not be possible. > > Julian, I'm not sure that Luigi is dealing with SMP on -current > rather than SMP on -stable. > > However, looking at the HEAD branch of /sys/i386/i386/locore.s, > I still see: > > ---------------------------------------------------------------------- > #ifdef SMP > /* > * Define layout of per-cpu address space. > * This is "constructed" in locore.s on the BSP and in mp_machdep.c > * for each AP. DO NOT REORDER THESE WITHOUT UPDATING THE REST! > */ > .globl SMP_prvspace, lapic > .set SMP_prvspace,(MPPTDI << PDRSHIFT) > .set lapic,SMP_prvspace + (NPTEPG-1) * PAGE_SIZE > #endif /* SMP */ > ... > #ifdef SMP > .globl cpu0prvpage > cpu0pp: .long 0 /* phys addr cpu0 private pg */ > cpu0prvpage: .long 0 /* relocated version */ > > .globl SMPpt > SMPptpa: .long 0 /* phys addr SMP page table */ > SMPpt: .long 0 /* relocated version */ > #endif /* SMP */ > ... > #ifdef SMP > .globl KPTphys > #endif > ... > #ifdef SMP > /* Allocate cpu0's private data page */ > ALLOCPAGES(1) > movl %esi,R(cpu0pp) > addl $KERNBASE, %esi > movl %esi, R(cpu0prvpage) /* relocated to KVM space */ > > /* Allocate SMP page table page */ > ALLOCPAGES(1) > movl %esi,R(SMPptpa) > addl $KERNBASE, %esi > movl %esi, R(SMPpt) /* relocated to KVM space */ > #endif /* SMP */ > ---------------------------------------------------------------------- > > Which indicates that it still exists. And it is wrong. Although it says 'private', it isn't in fact per-cpu. We just abuse the old code to cheat on allocating KVM. All cpus use the same identical "private" space. We use %fs exclusively for per-cpu data, but for various bogus reasons we still have this historical wart around. Cheers, -Peter -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the messagehelp
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20020807020033.1F0842A7D6>