From owner-svn-src-all@FreeBSD.ORG Sat Jun 19 04:57:59 2010 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 16B001065675; Sat, 19 Jun 2010 04:57:59 +0000 (UTC) (envelope-from marcel@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 048088FC12; Sat, 19 Jun 2010 04:57:59 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id o5J4vw9Y076800; Sat, 19 Jun 2010 04:57:58 GMT (envelope-from marcel@svn.freebsd.org) Received: (from marcel@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id o5J4vwu6076796; Sat, 19 Jun 2010 04:57:58 GMT (envelope-from marcel@svn.freebsd.org) Message-Id: <201006190457.o5J4vwu6076796@svn.freebsd.org> From: Marcel Moolenaar Date: Sat, 19 Jun 2010 04:57:58 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-8@freebsd.org X-SVN-Group: stable-8 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r209326 - in stable/8/sys/ia64: ia64 include X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Jun 2010 04:57:59 -0000 Author: marcel Date: Sat Jun 19 04:57:58 2010 New Revision: 209326 URL: http://svn.freebsd.org/changeset/base/209326 Log: MFC revs 209026 and 209085: o Bump MAX_BPAGES from 256 to 1024. o Synchronize the kernel entry on all CPUs with the use of the ptc.g instruction on a single CPU by implementing a bare-bones readers- writer lock. Modified: stable/8/sys/ia64/ia64/busdma_machdep.c stable/8/sys/ia64/ia64/exception.S stable/8/sys/ia64/ia64/pmap.c stable/8/sys/ia64/ia64/vm_machdep.c stable/8/sys/ia64/include/md_var.h Directory Properties: stable/8/sys/ (props changed) stable/8/sys/amd64/include/xen/ (props changed) stable/8/sys/cddl/contrib/opensolaris/ (props changed) stable/8/sys/contrib/dev/acpica/ (props changed) stable/8/sys/contrib/pf/ (props changed) stable/8/sys/dev/xen/xenpci/ (props changed) stable/8/sys/geom/sched/ (props changed) Modified: stable/8/sys/ia64/ia64/busdma_machdep.c ============================================================================== --- stable/8/sys/ia64/ia64/busdma_machdep.c Sat Jun 19 02:30:10 2010 (r209325) +++ stable/8/sys/ia64/ia64/busdma_machdep.c Sat Jun 19 04:57:58 2010 (r209326) @@ -48,7 +48,7 @@ __FBSDID("$FreeBSD$"); #include #include -#define MAX_BPAGES 256 +#define MAX_BPAGES 1024 struct bus_dma_tag { bus_dma_tag_t parent; @@ -77,7 +77,7 @@ struct bounce_page { STAILQ_ENTRY(bounce_page) links; }; -int busdma_swi_pending; +u_int busdma_swi_pending; static struct mtx bounce_lock; static STAILQ_HEAD(bp_list, bounce_page) bounce_page_list; Modified: stable/8/sys/ia64/ia64/exception.S ============================================================================== --- stable/8/sys/ia64/ia64/exception.S Sat Jun 19 02:30:10 2010 (r209325) +++ stable/8/sys/ia64/ia64/exception.S Sat Jun 19 04:57:58 2010 (r209326) @@ -170,6 +170,27 @@ ENTRY_NOPROFILE(exception_save, 0) * r30,r31=trapframe pointers * p14,p15=memory stack switch */ + + /* PTC.G enter non-exclusive */ + mov r24 = ar.ccv + movl r25 = pmap_ptc_g_sem + ;; +.ptc_g_0: + ld8.acq r26 = [r25] + ;; + tbit.nz p12, p0 = r26, 63 +(p12) br.cond.spnt.few .ptc_g_0 + ;; + mov ar.ccv = r26 + adds r27 = 1, r26 + ;; + cmpxchg8.rel r27 = [r25], r27, ar.ccv + ;; + cmp.ne p12, p0 = r26, r27 +(p12) br.cond.spnt.few .ptc_g_0 + ;; + mov ar.ccv = r24 + exception_save_restart: { .mmi st8 [r30]=r19,16 // length @@ -407,6 +428,23 @@ exception_save_restart: movl gp=__gp ;; } + + /* PTC.G leave non-exclusive */ + srlz.d + movl r25 = pmap_ptc_g_sem + ;; +.ptc_g_1: + ld8.acq r26 = [r25] + ;; + mov ar.ccv = r26 + adds r27 = -1, r26 + ;; + cmpxchg8.rel r27 = [r25], r27, ar.ccv + ;; + cmp.ne p12, p0 = r26, r27 +(p12) br.cond.spnt.few .ptc_g_1 + ;; + { .mib srlz.d nop 0 Modified: stable/8/sys/ia64/ia64/pmap.c ============================================================================== --- stable/8/sys/ia64/ia64/pmap.c Sat Jun 19 02:30:10 2010 (r209325) +++ stable/8/sys/ia64/ia64/pmap.c Sat Jun 19 04:57:58 2010 (r209326) @@ -182,7 +182,8 @@ static uint64_t pmap_ptc_e_count1 = 3; static uint64_t pmap_ptc_e_count2 = 2; static uint64_t pmap_ptc_e_stride1 = 0x2000; static uint64_t pmap_ptc_e_stride2 = 0x100000000; -struct mtx pmap_ptcmutex; + +volatile u_long pmap_ptc_g_sem; /* * Data for the RID allocator @@ -340,7 +341,6 @@ pmap_bootstrap() pmap_ptc_e_count2, pmap_ptc_e_stride1, pmap_ptc_e_stride2); - mtx_init(&pmap_ptcmutex, "Global PTC lock", NULL, MTX_SPIN); /* * Setup RIDs. RIDs 0..7 are reserved for the kernel. @@ -540,7 +540,8 @@ pmap_invalidate_page(vm_offset_t va) { struct ia64_lpte *pte; struct pcpu *pc; - uint64_t tag; + uint64_t tag, sem; + register_t is; u_int vhpt_ofs; critical_enter(); @@ -550,10 +551,32 @@ pmap_invalidate_page(vm_offset_t va) pte = (struct ia64_lpte *)(pc->pc_md.vhpt + vhpt_ofs); atomic_cmpset_64(&pte->tag, tag, 1UL << 63); } - critical_exit(); - mtx_lock_spin(&pmap_ptcmutex); + + /* PTC.G enter exclusive */ + is = intr_disable(); + + /* Atomically assert writer after all writers have gone. */ + do { + /* Wait until there's no more writer. */ + do { + sem = atomic_load_acq_long(&pmap_ptc_g_sem); + tag = sem | (1ul << 63); + } while (sem == tag); + } while (!atomic_cmpset_rel_long(&pmap_ptc_g_sem, sem, tag)); + + /* Wait until all readers are gone. */ + tag = (1ul << 63); + do { + sem = atomic_load_acq_long(&pmap_ptc_g_sem); + } while (sem != tag); + ia64_ptc_ga(va, PAGE_SHIFT << 2); - mtx_unlock_spin(&pmap_ptcmutex); + + /* PTC.G leave exclusive */ + atomic_store_rel_long(&pmap_ptc_g_sem, 0); + + intr_restore(is); + critical_exit(); } static void Modified: stable/8/sys/ia64/ia64/vm_machdep.c ============================================================================== --- stable/8/sys/ia64/ia64/vm_machdep.c Sat Jun 19 02:30:10 2010 (r209325) +++ stable/8/sys/ia64/ia64/vm_machdep.c Sat Jun 19 04:57:58 2010 (r209326) @@ -378,9 +378,8 @@ sf_buf_free(struct sf_buf *sf) */ void swi_vm(void *dummy) -{ -#if 0 +{ + if (busdma_swi_pending != 0) busdma_swi(); -#endif } Modified: stable/8/sys/ia64/include/md_var.h ============================================================================== --- stable/8/sys/ia64/include/md_var.h Sat Jun 19 02:30:10 2010 (r209325) +++ stable/8/sys/ia64/include/md_var.h Sat Jun 19 04:57:58 2010 (r209326) @@ -75,6 +75,7 @@ struct ia64_init_return { extern uint64_t ia64_lapic_addr; extern long Maxmem; +extern u_int busdma_swi_pending; void busdma_swi(void); int copyout_regstack(struct thread *, uint64_t *, uint64_t *);