From owner-svn-src-all@freebsd.org Fri Sep 28 17:03:49 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4F0A210B5087; Fri, 28 Sep 2018 17:03:49 +0000 (UTC) (envelope-from tijl@freebsd.org) Received: from mailrelay115.isp.belgacom.be (mailrelay115.isp.belgacom.be [195.238.20.142]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "relay.skynet.be", Issuer "GlobalSign Organization Validation CA - SHA256 - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 56BD77A62B; Fri, 28 Sep 2018 17:03:48 +0000 (UTC) (envelope-from tijl@freebsd.org) X-Belgacom-Dynamic: yes IronPort-PHdr: =?us-ascii?q?9a23=3A6FCYxBDKG7GSrSzRMUzuUyQJP3N1i/DPJgcQr6?= =?us-ascii?q?AfoPdwSPX+p8bcNUDSrc9gkEXOFd2Cra4c1KyO6+jJYi8p2d65qncMcZhBBV?= =?us-ascii?q?cuqP49uEgeOvODElDxN/XwbiY3T4xoXV5h+GynYwAOQJ6tL1LdrWev4jEMBx?= =?us-ascii?q?7xKRR6JvjvGo7Vks+7y/2+94fcbglUhjexe69+IAmrpgjNq8cahpdvJLwswR?= =?us-ascii?q?XTuHtIfOpWxWJsJV2Nmhv3+9m98p1+/SlOovwt78FPX7n0cKQ+VrxYES8pM3?= =?us-ascii?q?sp683xtBnMVhWA630BWWgLiBVIAgzF7BbnXpfttybxq+Rw1DWGMcDwULs5Xy?= =?us-ascii?q?mp4aV2Rx/ykCoJOT43/n/KhMJzjq1brh2uqABwzYPPfIGYNuZycr/Bcd8GW2?= =?us-ascii?q?ZMWNtaWSxbAoO7aosCF/QPP+Ncr4LguVYOqhy+BRKyC+Pq0DBIgGT21rA93u?= =?us-ascii?q?Q9EAHG3gggEskOvnTOstr6KrofX/27wqXSyjXDdfxW1C775YPVfB4hpvSMUq?= =?us-ascii?q?hxccrX0UQvGRnFgUuOpoP+JDOay/4Bs2+B7+pvU++klm0pqxlprzWsxMogkI?= =?us-ascii?q?jEi40PxlzZ8Sh0z5w5KcOlREN0Z9OvDYFeuDuAN4RsR8MvW2RouCEnxbIYoZ?= =?us-ascii?q?O7Zy0KyIg/xx7YdvyHb5CE4hL9W+aVJjd1nHFld6i7hxaz9Eiv1Pf8Vs6u3F?= =?us-ascii?q?ZWrypFkN3MumoR2BzU7ciHUeVy/kK71jaJzADf8OZEIVo7labDKp4hxKA/lo?= =?us-ascii?q?YLvEnAHCL6gkT7gaCMekk5/uWl5P7rbqv7qpKTL4N0jxvxMqUqmsyxG+Q4NQ?= =?us-ascii?q?0OUnCH+eS91r3j+Vb0QLtQjvIojKbWq4vaKtkBqq6+GQ9V3Z4v6wyjADe+zN?= =?us-ascii?q?QYgX4HIUpeeBKZiYjlIkrOIOrkAvumg1Sjji1kyO3dM73vGJXCMGbMn6z6fb?= =?us-ascii?q?lg8UJczxAzzd9H7ZJOFr4BOO7zWlP2tNHAFR85LQq0z//8CNVnyowSQHiAAq?= =?us-ascii?q?udMK7Jv1+I/PwgLPeXa48OpTnxMf8l5+ThjXUhg18SYbGp3YcLaHC/BvlmOF?= =?us-ascii?q?+Zbmb1jtgdFmcKuw8+TO32iF2eTzFTfW2yUrky5jEhFo2pE5zMRpuwj7Cb2y?= =?us-ascii?q?e7BJJWbHhcCl+QCXfoa5mEW/AUZSKRI89uiScJVba7S4A/yx6hrxT6xKd9Iu?= =?us-ascii?q?rT4C0YusGr6N8gxOrVjg078yB1CYy33mWKUnpzgCtcRD489I5lrEFX8XvF1r?= =?us-ascii?q?J31a92D9tWss+uVk8RMpnHwulzDcu6Dh7AfNOhZkyrT/+dLXc2VN1nkIxGWF?= =?us-ascii?q?p0B9j31kOL5CGtGbJAz7E=3D?= X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: =?us-ascii?q?A2AVAABLXq5b/9bCQFdbGQEBAQEBAQE?= =?us-ascii?q?BAQEBAQcBAQEBAQGBUoINZm0SFROMaItSAQGCDDMBhx2CAI0FgXouhEkCg3o?= =?us-ascii?q?iNRcBAwEBAgEBAgFsHAxCDAGBZiKCYgEFOhwjEAsOBgQJGgsPKh4GE4MiggU?= =?us-ascii?q?LpgGEKwGEHoFBBYsVggCEJIR+hVgCjkeOE0MJhkOJYiKPSIwGimABNoFVTTA?= =?us-ascii?q?IgyeCJRcRiEmFQD0wg0KJewEB?= X-IPAS-Result: =?us-ascii?q?A2AVAABLXq5b/9bCQFdbGQEBAQEBAQEBAQEBAQcBAQEBA?= =?us-ascii?q?QGBUoINZm0SFROMaItSAQGCDDMBhx2CAI0FgXouhEkCg3oiNRcBAwEBAgEBA?= =?us-ascii?q?gFsHAxCDAGBZiKCYgEFOhwjEAsOBgQJGgsPKh4GE4MiggULpgGEKwGEHoFBB?= =?us-ascii?q?YsVggCEJIR+hVgCjkeOE0MJhkOJYiKPSIwGimABNoFVTTAIgyeCJRcRiEmFQ?= =?us-ascii?q?D0wg0KJewEB?= Received: from 214.194-64-87.adsl-dyn.isp.belgacom.be (HELO kalimero.tijl.coosemans.org) ([87.64.194.214]) by relay.skynet.be with ESMTP; 28 Sep 2018 19:02:35 +0200 Received: from kalimero.tijl.coosemans.org (kalimero.tijl.coosemans.org [127.0.0.1]) by kalimero.tijl.coosemans.org (8.15.2/8.15.2) with ESMTP id w8SH2YVo002425; Fri, 28 Sep 2018 19:02:35 +0200 (CEST) (envelope-from tijl@FreeBSD.org) Date: Fri, 28 Sep 2018 19:02:34 +0200 From: =?UTF-8?B?VMSzbA==?= Coosemans To: Konstantin Belousov Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r338807 - in head/sys: amd64/amd64 amd64/include dev/drm2 dev/drm2/i915 i386/i386 i386/include x86/iommu Message-ID: <20180928190234.53240b03@kalimero.tijl.coosemans.org> In-Reply-To: <201809191935.w8JJZ2ob054223@repo.freebsd.org> References: <201809191935.w8JJZ2ob054223@repo.freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Sep 2018 17:03:49 -0000 On Wed, 19 Sep 2018 19:35:02 +0000 (UTC) Konstantin Belousov wrote: > Author: kib > Date: Wed Sep 19 19:35:02 2018 > New Revision: 338807 > URL: https://svnweb.freebsd.org/changeset/base/338807 > > Log: > Convert x86 cache invalidation functions to ifuncs. > > This simplifies the runtime logic and reduces the number of > runtime-constant branches. > > Reviewed by: alc, markj > Sponsored by: The FreeBSD Foundation > Approved by: re (gjb) > Differential revision: https://reviews.freebsd.org/D16736 > > Modified: > head/sys/amd64/amd64/pmap.c > head/sys/amd64/include/pmap.h > head/sys/dev/drm2/drm_os_freebsd.c > head/sys/dev/drm2/i915/intel_ringbuffer.c > head/sys/i386/i386/pmap.c > head/sys/i386/i386/vm_machdep.c > head/sys/i386/include/pmap.h > head/sys/x86/iommu/intel_utils.c > > Modified: head/sys/i386/i386/pmap.c > ============================================================================== > --- head/sys/i386/i386/pmap.c Wed Sep 19 19:13:43 2018 (r338806) > +++ head/sys/i386/i386/pmap.c Wed Sep 19 19:35:02 2018 (r338807) > @@ -148,6 +148,7 @@ __FBSDID("$FreeBSD$"); > #include > #include > #endif > +#include > #include > #include > #include > @@ -314,6 +315,10 @@ static vm_page_t pmap_enter_quick_locked(pmap_t pmap, > vm_page_t m, vm_prot_t prot, vm_page_t mpte); > static void pmap_flush_page(vm_page_t m); > static int pmap_insert_pt_page(pmap_t pmap, vm_page_t mpte); > +static void pmap_invalidate_cache_range_selfsnoop(vm_offset_t sva, > + vm_offset_t eva); > +static void pmap_invalidate_cache_range_all(vm_offset_t sva, > + vm_offset_t eva); > static void pmap_invalidate_pde_page(pmap_t pmap, vm_offset_t va, > pd_entry_t pde); > static void pmap_fill_ptp(pt_entry_t *firstpte, pt_entry_t newpte); > @@ -1407,37 +1412,62 @@ pmap_invalidate_pde_page(pmap_t pmap, vm_offset_t va, > pmap_invalidate_page(pmap, va); > } > > +DEFINE_IFUNC(, void, pmap_invalidate_cache_range, (vm_offset_t, vm_offset_t), > + static) > +{ > + > + if ((cpu_feature & CPUID_SS) != 0) > + return (pmap_invalidate_cache_range_selfsnoop); > + if ((cpu_feature & CPUID_CLFSH) != 0) > + return (pmap_force_invalidate_cache_range); > + return (pmap_invalidate_cache_range_all); > +} > + > #define PMAP_CLFLUSH_THRESHOLD (2 * 1024 * 1024) > > +static void > +pmap_invalidate_cache_range_check_align(vm_offset_t sva, vm_offset_t eva) > +{ > + > + KASSERT((sva & PAGE_MASK) == 0, > + ("pmap_invalidate_cache_range: sva not page-aligned")); > + KASSERT((eva & PAGE_MASK) == 0, > + ("pmap_invalidate_cache_range: eva not page-aligned")); > +} > + > +static void > +pmap_invalidate_cache_range_selfsnoop(vm_offset_t sva, vm_offset_t eva) > +{ > + > + pmap_invalidate_cache_range_check_align(sva, eva); > +} > + > void > -pmap_invalidate_cache_range(vm_offset_t sva, vm_offset_t eva, boolean_t force) > +pmap_force_invalidate_cache_range(vm_offset_t sva, vm_offset_t eva) > { > > - if (force) { > - sva &= ~(vm_offset_t)(cpu_clflush_line_size - 1); > - } else { > - KASSERT((sva & PAGE_MASK) == 0, > - ("pmap_invalidate_cache_range: sva not page-aligned")); > - KASSERT((eva & PAGE_MASK) == 0, > - ("pmap_invalidate_cache_range: eva not page-aligned")); > + sva &= ~(vm_offset_t)(cpu_clflush_line_size - 1); > + if (eva - sva >= PMAP_CLFLUSH_THRESHOLD) { > + /* > + * The supplied range is bigger than 2MB. > + * Globally invalidate cache. > + */ > + pmap_invalidate_cache(); > + return; > } > > - if ((cpu_feature & CPUID_SS) != 0 && !force) > - ; /* If "Self Snoop" is supported and allowed, do nothing. */ > - else if ((cpu_stdext_feature & CPUID_STDEXT_CLFLUSHOPT) != 0 && > - eva - sva < PMAP_CLFLUSH_THRESHOLD) { > -#ifdef DEV_APIC > + /* > + * XXX: Some CPUs fault, hang, or trash the local APIC > + * registers if we use CLFLUSH on the local APIC > + * range. The local APIC is always uncached, so we > + * don't need to flush for that range anyway. > + */ > + if (pmap_kextract(sva) == lapic_paddr) > + return; > + > + if ((cpu_stdext_feature & CPUID_STDEXT_CLFLUSHOPT) != 0) { > /* > - * XXX: Some CPUs fault, hang, or trash the local APIC > - * registers if we use CLFLUSH on the local APIC > - * range. The local APIC is always uncached, so we > - * don't need to flush for that range anyway. > - */ > - if (pmap_kextract(sva) == lapic_paddr) > - return; > -#endif > - /* > - * Otherwise, do per-cache line flush. Use the sfence > + * Do per-cache line flush. Use the sfence > * instruction to insure that previous stores are > * included in the write-back. The processor > * propagates flush to other processors in the cache > @@ -1447,12 +1477,7 @@ pmap_invalidate_cache_range(vm_offset_t sva, vm_offset > for (; sva < eva; sva += cpu_clflush_line_size) > clflushopt(sva); > sfence(); > - } else if ((cpu_feature & CPUID_CLFSH) != 0 && > - eva - sva < PMAP_CLFLUSH_THRESHOLD) { > -#ifdef DEV_APIC > - if (pmap_kextract(sva) == lapic_paddr) > - return; > -#endif > + } else { > /* > * Writes are ordered by CLFLUSH on Intel CPUs. > */ The removal of #ifdef DEV_APIC breaks building kernels without device apic: /usr/src/sys/i386/i386/pmap.c:1465:28: error: use of undeclared identifier 'lapic_paddr' if (pmap_kextract(sva) == lapic_paddr) ^ 1 error generated.