From owner-cvs-src@FreeBSD.ORG Sat Aug 6 20:28:20 2005 Return-Path: X-Original-To: cvs-src@FreeBSD.org Delivered-To: cvs-src@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8F08416A421; Sat, 6 Aug 2005 20:28:20 +0000 (GMT) (envelope-from marcel@FreeBSD.org) Received: from repoman.freebsd.org (repoman.freebsd.org [216.136.204.115]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1B26243D49; Sat, 6 Aug 2005 20:28:20 +0000 (GMT) (envelope-from marcel@FreeBSD.org) Received: from repoman.freebsd.org (localhost [127.0.0.1]) by repoman.freebsd.org (8.13.1/8.13.1) with ESMTP id j76KSJ7k019033; Sat, 6 Aug 2005 20:28:19 GMT (envelope-from marcel@repoman.freebsd.org) Received: (from marcel@localhost) by repoman.freebsd.org (8.13.1/8.13.1/Submit) id j76KSJtM019032; Sat, 6 Aug 2005 20:28:19 GMT (envelope-from marcel) Message-Id: <200508062028.j76KSJtM019032@repoman.freebsd.org> From: Marcel Moolenaar Date: Sat, 6 Aug 2005 20:28:19 +0000 (UTC) To: src-committers@FreeBSD.org, cvs-src@FreeBSD.org, cvs-all@FreeBSD.org X-FreeBSD-CVS-Branch: HEAD Cc: Subject: cvs commit: src/sys/ia64/ia64 exception.S interrupt.c machdep.c mp_machdep.c pmap.c trap.c vm_machdep.c src/sys/ia64/include proc.h smp.h X-BeenThere: cvs-src@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: CVS commit messages for the src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Aug 2005 20:28:20 -0000 marcel 2005-08-06 20:28:19 UTC FreeBSD src repository Modified files: sys/ia64/ia64 exception.S interrupt.c machdep.c mp_machdep.c pmap.c trap.c vm_machdep.c sys/ia64/include proc.h smp.h Log: Improve SMP support: o Allocate a VHPT per CPU. The VHPT is a hash table that the CPU uses to look up translations it can't find in the TLB. As such, the VHPT serves as a level 1 cache (the TLB being a level 0 cache) and best results are obtained when it's not shared between CPUs. The collision chain (i.e. the hash bucket) is shared between CPUs, as all buckets together constitute our collection of PTEs. To achieve this, the collision chain does not point to the first PTE in the list anymore, but to a hash bucket head structure. The head structure contains the pointer to the first PTE in the list, as well as a mutex to lock the bucket. Thus, each bucket is locked independently of each other. With at least 1024 buckets in the VHPT, this provides for sufficiently finei-grained locking to make the ssolution scalable to large SMP machines. o Add synchronisation to the lazy FP context switching. We do this with a seperate per-thread lock. On SMP machines the lazy high FP context switching without synchronisation caused inconsistent state, which resulted in a panic. Since the use of the high FP registers is not common, it's possible that races exist. The ia64 package build has proven to be a good stress test, so this will get plenty of exercise in the near future. o Don't use the local ID of the processor we want to send the IPI to as the argument to ipi_send(). use the struct pcpu pointer instead. The reason for this is that IPI delivery is unreliable. It has been observed that sending an IPI to a CPU causes it to receive a stray external interrupt. As such, we need a way to make the delivery reliable. The intended solution is to queue requests in the target CPU's per-CPU structure and use a single IPI to inform the CPU that there's a new entry in the queue. If that IPI gets lost, the CPU can check it's queue at any convenient time (such as for each clock interrupt). This also allows us to send requests to a CPU without interrupting it, if such would be beneficial. With these changes SMP is almost working. There are still some random process crashes and the machine can hang due to having the IPI lost that deals with the high FP context switch. The overhead of introducing the hash bucket head structure results in a performance degradation of about 1% for UP (extra pointer indirection). This is surprisingly small and is offset by gaining reasonably/good scalable SMP support. Revision Changes Path 1.57 +8 -0 src/sys/ia64/ia64/exception.S 1.50 +5 -0 src/sys/ia64/ia64/interrupt.c 1.201 +30 -13 src/sys/ia64/ia64/machdep.c 1.56 +29 -25 src/sys/ia64/ia64/mp_machdep.c 1.161 +227 -272 src/sys/ia64/ia64/pmap.c 1.114 +12 -7 src/sys/ia64/ia64/trap.c 1.91 +1 -0 src/sys/ia64/ia64/vm_machdep.c 1.15 +2 -1 src/sys/ia64/include/proc.h 1.10 +4 -2 src/sys/ia64/include/smp.h