From owner-freebsd-arch@FreeBSD.ORG Mon Aug 18 22:35:53 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C64FAFD1; Mon, 18 Aug 2014 22:35:53 +0000 (UTC) Received: from mail-ie0-x22e.google.com (mail-ie0-x22e.google.com [IPv6:2607:f8b0:4001:c03::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 88F883F43; Mon, 18 Aug 2014 22:35:53 +0000 (UTC) Received: by mail-ie0-f174.google.com with SMTP id rp18so176754iec.5 for ; Mon, 18 Aug 2014 15:35:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=f5LlijFXX7pDpvpWeF6NZZMfcBpCCrtSa9MFRU69x3Y=; b=DFW1mSqsSRI1uFdfrYZDbmQBi+Mx9mf0220Vhq7yvK7Z85n2g6+WDSslneqcjwq35x nk+/DTSIO+RD3w38eu3uSZhsuyRHGYLQpl8unnB6+YSPzfWuiJPETJsGQk7F3DqryLP7 JwAm9jDQXaKUl7RmPVEXSuYKAFV+Yzr+Q2jc+PDsw/vAu6RbJmr7eEKuxFHU7nPZkz/4 2bT1S3rLrRg0ROEiYqo+5ot0zzgAj3WdbePnldg+6Wrdm3J8i20XCDoEGYmWFCcwaqoc W57jSrAVxyOPG55MvQq+SSCZoVGUhPMl+0BEEclylHD2rJTq+tbl37/j38ISM3hkobZl oOhQ== MIME-Version: 1.0 X-Received: by 10.43.127.136 with SMTP id ha8mr3526994icc.78.1408401352808; Mon, 18 Aug 2014 15:35:52 -0700 (PDT) Received: by 10.43.17.196 with HTTP; Mon, 18 Aug 2014 15:35:52 -0700 (PDT) Reply-To: alc@freebsd.org In-Reply-To: <257A0976-7C5E-4029-AF32-BFB3A6C60832@bsdimp.com> References: <53F215A9.8010708@FreeBSD.org> <20140818183925.GP2737@kib.kiev.ua> <53F25E60.5050109@freebsd.org> <257A0976-7C5E-4029-AF32-BFB3A6C60832@bsdimp.com> Date: Mon, 18 Aug 2014 17:35:52 -0500 Message-ID: Subject: Re: superpages for UMA From: Alan Cox To: Warner Losh Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: "freebsd-arch@freebsd.org" , Peter Grehan X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Aug 2014 22:35:53 -0000 On Mon, Aug 18, 2014 at 3:26 PM, Warner Losh wrote: > > On Aug 18, 2014, at 2:13 PM, Peter Grehan wrote: > > >> Newer Intel CPUs have more entries, and AMD CPUs have long (since > >> Barcelona) had more. In particular, they allow 2 MB page mappings to = be > >> cached in a larger L2 TLB. Nowadays, the trouble is with the 1 GB > pages. > >> A lot of CPUs still only support an 8 entry, 1 level TLB for 1 GB page= s. > > > > There are new(ish) ones effectively without 1GB pages. From the > "Software Optimization Guide for AMD Family 16h Processors" > > > > "Smashing" > > ... > > "when the Family 16h processor encounters a 1-Gbyte page size, it will > smash translations of that 1-Gbyte region into 2-Mbyte TLB entries, each > > of which translates a 2-Mbyte region of the 1-Gbyte page." > > =E2=80=9Cwe=E2=80=99ll emulate this feature designed to make things go fa= ster in hardware > in software by doing the very thing that makes it go slow in hardware.=E2= =80=9D > > Fun times. Performance Smashing! > > I'm guessing that these are low-power processors, where they don't want to have another CAM consuming power. Under those circumstances, it's still better to support 1 GB page mappings in the page table even if the TLB doesn't support them than not to support 1 GB page mappings at all. With the hierarchical page tables on x86, you get a 512x reduction in page table size with each increase in page size. So, on a TLB miss, the page table walk is more likely to be all L2 data cache hits, rather than misses that go all the way to DRAM. One feature that I always liked about the AMD performance counters was that they allowed you to count L2 cache misses caused by page table walks on a TLB miss. This was often a better predictor of whether large pages were going to be beneficial than counting TLB misses.