From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 5 15:14:24 2013 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 0DF3068C; Tue, 5 Feb 2013 15:14:24 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id 5E691F76; Tue, 5 Feb 2013 15:14:23 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.6/8.14.6) with ESMTP id r15FEDQQ060971; Tue, 5 Feb 2013 17:14:13 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.7.4 kib.kiev.ua r15FEDQQ060971 Received: (from kostik@localhost) by tom.home (8.14.6/8.14.6/Submit) id r15FEDgt060970; Tue, 5 Feb 2013 17:14:13 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 5 Feb 2013 17:14:13 +0200 From: Konstantin Belousov To: Neel Natu Subject: Re: dynamically calculating NKPT [was: Re: huge ktr buffer] Message-ID: <20130205151413.GL2522@kib.kiev.ua> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="oImBTl0TNA0mSDFD" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: alc@freebsd.org, davide@freebsd.org, hackers@freebsd.org, avg@freebsd.org, rank1seeker@gmail.com X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Feb 2013 15:14:24 -0000 --oImBTl0TNA0mSDFD Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Feb 04, 2013 at 03:05:15PM -0800, Neel Natu wrote: > Hi, >=20 > I have a patch to dynamically calculate NKPT for amd64 kernels. This > should fix the various issues that people pointed out in the email > thread. >=20 > Please review and let me know if there are any objections to committing t= his. >=20 > Also, thanks to Alan (alc@) for reviewing and providing feedback on > the initial version of the patch. >=20 > Patch (also available at http://people.freebsd.org/~neel/patches/nkpt_dif= f.txt): >=20 > Index: sys/amd64/include/pmap.h > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/amd64/include/pmap.h (revision 246277) > +++ sys/amd64/include/pmap.h (working copy) > @@ -113,13 +113,7 @@ > ((unsigned long)(l2) << PDRSHIFT) | \ > ((unsigned long)(l1) << PAGE_SHIFT)) >=20 > -/* Initial number of kernel page tables. */ > -#ifndef NKPT > -#define NKPT 32 > -#endif > - > #define NKPML4E 1 /* number of kernel PML4 slots */ > -#define NKPDPE howmany(NKPT, NPDEPG)/* number of kernel PDP slots */ >=20 > #define NUPML4E (NPML4EPG/2) /* number of userland PML4 pages */ > #define NUPDPE (NUPML4E*NPDPEPG)/* number of userland PDP pages */ > @@ -181,6 +175,7 @@ > #define PML4map ((pd_entry_t *)(addr_PML4map)) > #define PML4pml4e ((pd_entry_t *)(addr_PML4pml4e)) >=20 > +extern int nkpt; /* Initial number of kernel page tables */ > extern u_int64_t KPDPphys; /* physical address of kernel level 3 */ > extern u_int64_t KPML4phys; /* physical address of kernel level 4 */ >=20 > Index: sys/amd64/amd64/minidump_machdep.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/amd64/amd64/minidump_machdep.c (revision 246277) > +++ sys/amd64/amd64/minidump_machdep.c (working copy) > @@ -232,7 +232,7 @@ > /* Walk page table pages, set bits in vm_page_dump */ > pmapsize =3D 0; > pdp =3D (uint64_t *)PHYS_TO_DMAP(KPDPphys); > - for (va =3D VM_MIN_KERNEL_ADDRESS; va < MAX(KERNBASE + NKPT * NBPDR, > + for (va =3D VM_MIN_KERNEL_ADDRESS; va < MAX(KERNBASE + nkpt * NBPDR, > kernel_vm_end); ) { > /* > * We always write a page, even if it is zero. Each > @@ -364,7 +364,7 @@ > /* Dump kernel page directory pages */ > bzero(fakepd, sizeof(fakepd)); > pdp =3D (uint64_t *)PHYS_TO_DMAP(KPDPphys); > - for (va =3D VM_MIN_KERNEL_ADDRESS; va < MAX(KERNBASE + NKPT * NBPDR, > + for (va =3D VM_MIN_KERNEL_ADDRESS; va < MAX(KERNBASE + nkpt * NBPDR, > kernel_vm_end); va +=3D NBPDP) { > i =3D (va >> PDPSHIFT) & ((1ul << NPDPEPGSHIFT) - 1); >=20 > Index: sys/amd64/amd64/pmap.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/amd64/amd64/pmap.c (revision 246277) > +++ sys/amd64/amd64/pmap.c (working copy) > @@ -202,6 +202,10 @@ > vm_offset_t virtual_avail; /* VA of first avail page (after kernel bss) = */ > vm_offset_t virtual_end; /* VA of last avail page (end of kernel AS) */ >=20 > +int nkpt; > +SYSCTL_INT(_machdep, OID_AUTO, nkpt, CTLFLAG_RD, &nkpt, 0, > + "Number of kernel page table pages allocated on bootup"); > + > static int ndmpdp; > static vm_paddr_t dmaplimit; > vm_offset_t kernel_vm_end =3D VM_MIN_KERNEL_ADDRESS; > @@ -495,17 +499,42 @@ >=20 > CTASSERT(powerof2(NDMPML4E)); >=20 > +/* number of kernel PDP slots */ > +#define NKPDPE(ptpgs) howmany((ptpgs), NPDEPG) > + > static void > +nkpt_init(vm_paddr_t addr) > +{ > + int pt_pages; > +=09 > +#ifdef NKPT > + pt_pages =3D NKPT; > +#else > + pt_pages =3D howmany(addr, 1 << PDRSHIFT); > + pt_pages +=3D NKPDPE(pt_pages); > + > + /* > + * Add some slop beyond the bare minimum required for bootstrapping > + * the kernel. > + * > + * This is quite important when allocating KVA for kernel modules. > + * The modules are required to be linked in the negative 2GB of > + * the address space. If we run out of KVA in this region then > + * pmap_growkernel() will need to allocate page table pages to map > + * the entire 512GB of KVA space which is an unnecessary tax on > + * physical memory. > + */ > + pt_pages +=3D 4; /* 8MB additional slop for kernel modules */ 8MB might be to low. I just checked one of my machines with fully modularized kernel, it takes slightly more than 6 MB to load 50 modules. I think that 16MB would be safer, but it probably needs to be scaled down based on the available phys memory. amd64 kernel could be booted on 128MB machine still. --oImBTl0TNA0mSDFD Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBAgAGBQJRESHEAAoJEJDCuSvBvK1BokcP/3oUV+2JJu9FdumDYQlPlkk4 jciIQpb/tl/Z/J9DJf6vThdOaw3R2QXhh1JrvkQFONTno2USeUJWivz7Rtvfdluq n200D5RsgkiWEBuBBLSE5PdKiioMePGFhuRed+67ISxgYWdC+5ZXXwvjHivdN52u +bDgV9d9D1iOX17Fcxu/yAlI5Aed1mlJw4o5YsQCnhw/vzXi2e0/gidqvbX+5JpM 42g8D5V35RWj+xtUvFDuGcFq0aGME0JMmJ/T9txIsWAawgZFqWM5gOVNESgtsLXc 82SWA6jnLy+/Vs889vQVDD6jVq3qIu7S4CnDAcXClfzCX172K6ImTiORAGkQhcoc mKZCEqPB7QyKic4N0jFVI9PdAeMhSzC9NJLxMIBvs7RasmU1QzMaMRpEfNPKEKiv +uG29qatSC3HaxEbmeK/Ix12RRnry9DJGUimZk6qiya3/rZAGFBBv425bQwNMe12 6rSJzUS1zR+Gus519xCqMs6Gxcn35qk+7gSnblsWNiVxRuDqTqodu/spfzRNYZ4I VfDZc0wG/aeeMPWwqLzAzzbWcZLuVaVN2nuN+Em2sVVy0YL/0QVmtSg7XmRJmYnX oqedILnMerIADtpQme5Rr7mh0j4bLcPeBJdYe34jVQqqk/9dzHOPoYU9N0Fsy5H5 NCfa+nr1IEM75JuU1h/G =N9W3 -----END PGP SIGNATURE----- --oImBTl0TNA0mSDFD--