From owner-freebsd-arm@FreeBSD.ORG Tue Mar 18 13:25:57 2014 Return-Path: Delivered-To: freebsd-arm@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2A6F04A4 for ; Tue, 18 Mar 2014 13:25:57 +0000 (UTC) Received: from mho-02-ewr.mailhop.org (mho-02-ewr.mailhop.org [204.13.248.72]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id DD648FCA for ; Tue, 18 Mar 2014 13:25:56 +0000 (UTC) Received: from c-24-8-230-52.hsd1.co.comcast.net ([24.8.230.52] helo=damnhippie.dyndns.org) by mho-02-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1WPu1e-000Jhu-HE; Tue, 18 Mar 2014 13:25:50 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id s2IDPkqe068474; Tue, 18 Mar 2014 07:25:46 -0600 (MDT) (envelope-from ian@FreeBSD.org) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 24.8.230.52 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX19JtA+998agMY/eYKX7VlxV Subject: Re: arm SMP on Cortex-A15 From: Ian Lepore To: Wojciech Macek In-Reply-To: References: <20131220125638.GA5132@mail.bsdpad.com> <20131222092913.GA89153@mail.bsdpad.com> <20131222123636.GA61193@ci0.org> Content-Type: multipart/mixed; boundary="=-n06oNHLaZG3WXWB+YZ+r" Date: Tue, 18 Mar 2014 07:25:46 -0600 Message-ID: <1395149146.1149.586.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Cc: freebsd-arm@FreeBSD.org X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Mar 2014 13:25:57 -0000 --=-n06oNHLaZG3WXWB+YZ+r Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit On Mon, 2014-03-17 at 09:29 +0100, Wojciech Macek wrote: > Hi, > > Finally I've found some time to continue SMP hacking. It seems that I > isolated the tlb/pmam failures and developed two simple patches that help. > There are still some pmap changes and TEX remap left, but I don't want to > use them now. > https://drive.google.com/folderview?id=0B-7yTLrPxaWtSzZPUGgtM3pnUjg&usp=sharing > * 01 - ensure that TTB is set before TLB invalidation and flush BTB to > comply the specs > * 02 - add missing TLB invalidations to pmap and fix invalidation order > > I chose buildworld -j4 as a stresstest, and run it on Arndale (USB rootfs) > and a different 4-core a15 chip (SATA rootfs). On both setups test passed > and was significantly faster than the one with previous patchset. > > I'd like to submit these changes to FreeBSD tree (with some help from our > local committers), so any comments and testing are really appreciated. > > Best regards, > Wojtek The first patch looks fine and is working without any problems for me. For the second patch, I propose the attached similar patch which combines your changes with some I got from Olivier. The main differences are moving the tlb flush outside the loop when propagating a change to all L1s, and moving the tlb flush (rather than adding another) in pmap_kenter_internal(). I believe even with the second patch there may still be some missing tlb flushes. -- Ian --=-n06oNHLaZG3WXWB+YZ+r Content-Disposition: inline; filename="smp_patch_02a.patch" Content-Type: text/x-patch; name="smp_patch_02a.patch"; charset="us-ascii" Content-Transfer-Encoding: 7bit Index: sys/arm/arm/pmap-v6.c =================================================================== --- sys/arm/arm/pmap-v6.c (revision 263054) +++ sys/arm/arm/pmap-v6.c (working copy) @@ -2130,6 +2130,8 @@ pmap_grow_l2_bucket(pmap_t pmap, vm_offset_t va) L1_C_PROTO; PTE_SYNC(pl1pd); } + cpu_tlb_flushID_SE(va); + cpu_cpwait(); return (l2b); } @@ -2387,14 +2389,6 @@ pmap_kenter_internal(vm_offset_t va, vm_offset_t p ptep = &l2b->l2b_kva[l2pte_index(va)]; opte = *ptep; - if (l2pte_valid(opte)) { - cpu_tlb_flushD_SE(va); - cpu_cpwait(); - } else { - if (opte == 0) - l2b->l2b_occupancy++; - } - if (flags & KENTER_CACHE) { *ptep = L2_S_PROTO | pa | pte_l2_s_cache_mode | L2_S_REF; pmap_set_prot(ptep, VM_PROT_READ | VM_PROT_WRITE, @@ -2405,10 +2399,17 @@ pmap_kenter_internal(vm_offset_t va, vm_offset_t p 0); } + PTE_SYNC(ptep); + if (l2pte_valid(opte)) { + cpu_tlb_flushD_SE(va); + } else { + if (opte == 0) + l2b->l2b_occupancy++; + } + cpu_cpwait(); + PDEBUG(1, printf("pmap_kenter: pte = %08x, opte = %08x, npte = %08x\n", (uint32_t) ptep, opte, *ptep)); - PTE_SYNC(ptep); - cpu_cpwait(); } void @@ -2474,10 +2475,10 @@ pmap_kremove(vm_offset_t va) opte = *ptep; if (l2pte_valid(opte)) { va = va & ~PAGE_MASK; + *ptep = 0; + PTE_SYNC(ptep); cpu_tlb_flushD_SE(va); cpu_cpwait(); - *ptep = 0; - PTE_SYNC(ptep); } } @@ -4380,6 +4381,8 @@ pmap_remove(pmap_t pmap, vm_offset_t sva, vm_offse } } + *ptep = 0; + PTE_SYNC(ptep); if (pmap_is_current(pmap)) { total++; if (total < PMAP_REMOVE_CLEAN_LIST_SIZE) { @@ -4390,8 +4393,6 @@ pmap_remove(pmap_t pmap, vm_offset_t sva, vm_offse } else if (total == PMAP_REMOVE_CLEAN_LIST_SIZE) flushall = 1; } - *ptep = 0; - PTE_SYNC(ptep); sva += PAGE_SIZE; ptep++; --=-n06oNHLaZG3WXWB+YZ+r--