From owner-freebsd-hackers@FreeBSD.ORG  Mon Jul 15 00:51:22 2013
Return-Path: <owner-freebsd-hackers@FreeBSD.ORG>
Delivered-To: freebsd-hackers@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
 by hub.freebsd.org (Postfix) with ESMTP id F335AC66;
 Mon, 15 Jul 2013 00:51:21 +0000 (UTC) (envelope-from torek@torek.net)
Received: from elf.torek.net (50-73-42-1-utah.hfc.comcastbusiness.net
 [50.73.42.1]) by mx1.freebsd.org (Postfix) with ESMTP id CD826B83;
 Mon, 15 Jul 2013 00:51:21 +0000 (UTC)
Received: from elf.torek.net (localhost [127.0.0.1])
 by elf.torek.net (8.14.5/8.14.5) with ESMTP id r6F0pEKj032863;
 Sun, 14 Jul 2013 18:51:14 -0600 (MDT) (envelope-from torek@torek.net)
Message-Id: <201307150051.r6F0pEKj032863@elf.torek.net>
From: Chris Torek <torek@torek.net>
To: Neel Natu <neelnatu@gmail.com>
Subject: Re: expanding amd64 past the 1TB limit
In-reply-to: Your message of "Thu, 11 Jul 2013 00:41:38 -0700."
 <CAFgRE9HycyTL-Mch9KF08Df6qFzzCTb-aU0AkKQhi7F5gtGevA@mail.gmail.com>
Date: Sun, 14 Jul 2013 18:51:14 -0600
X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3
 (elf.torek.net [127.0.0.1]); Sun, 14 Jul 2013 18:51:14 -0600 (MDT)
Cc: Konstantin Belousov <kostikbel@gmail.com>, freebsd-hackers@freebsd.org,
 kib@freebsd.org, alc@freebsd.org
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
 <freebsd-hackers.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-hackers>, 
 <mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
 <mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 15 Jul 2013 00:51:22 -0000

>The changes associated with pt_p, pd_p and p4_p are cosmetic and IMHO
>detract from the meat of the change.
>
>My preference would be for the cosmetic changes to be committed
>separately from the changes that rearrange the KVA.

I can split these out.

>> -       DMPDPphys = allocpages(firstaddr, NDMPML4E);
>> +       ndmpdpphys = howmany(ndmpdp, NPML4EPG);
>
>NPDPEPG should be used here instead of NPML4EPG even though they are
>numerically identical.

Ah, late-night editing thinko. :-)

>> + * KERNBASE.  Similarly, if KMPL4I < (base+N), extra level 4 PDEs are
>
>level 2 PDE's, right?

Er, yes (and L1s of course), and that should be < (base+N-1).

The idea here is that if for some reason KERNBASE were "in the
middle" of the NKPML4E entries, you'd have, at least initially
when only a few things are valid at VM_MIN_ADDRESS and at
KERNBASE:

             L4 table    L3 page              ... mapping:
            +-------+                         VM_MIN_KERNEL_ADDRESS
  KPMLBASE  |   * ---> [valid at 0 --> L2, then all-0s]
            +-------+
            |   * ---> [all-0s]
            +-------+
            |   * ---> [all-0s]
            +-------+
            |  ...  |
            +-------+                         KERNBASE
    KPML4I  |   * ---> [mostly-0s; the KDPDI'th --> L2]
            +-------+
  base+N-1  |   * ---> [all-0s]
            +-------+                         VM_MAX_KERNEL_ADDRESS

but then eventually allocating something between KERNBASE and
VM_MAX_KERNEL_ADDRESS would cause that last page of "all-0s" to
get filled in, etc.  Of course KPML4I should be the very last
(511'th) L4 table entry always, unless the kernel grows really
huge :-)

I'll fix the comments in the split-up patch.

Chris