Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 22 Apr 2002 19:14:56 -0700
From:      Terry Lambert <tlambert2@mindspring.com>
To:        David Schultz <dschultz@uclink.Berkeley.EDU>
Cc:        freebsd-current@FreeBSD.ORG, freebsd-stable@FreeBSD.ORG
Subject:   Re: FreeBSD 4.5-STABLE not easily scalable to large servers ... ?
Message-ID:  <3CC4C3A0.B1279BFC@mindspring.com>
References:  <20020421191440.J1721-100000@mail1.hub.org> <3CC3D494.649C2A8E@mindspring.com> <20020422060952.B31888@HAL9000.wox.org>

next in thread | previous in thread | raw e-mail | index | archive | help
David Schultz wrote:
> Thus spake Terry Lambert <tlambert2@mindspring.com>:
> > If you want more, then you need to use a 64 bit processor (or use a
> > processor that supports bank selection, and hack up FreeBSD to do
> > bank swapping on 2G at a time, just like Linux has been hacked up,
> > and expect that it won't be very useful).
> 
> I'm guessing that this just means looking at more than 4 GB of memory
> by working with 2 GB frames at a time.  As I recall, David Greenman
> said that this hack would essentially require a rewrite of the VM
> system.  Does this just boil down to using 36 bit physical addresses?
> Are there plans for FreeBSD to support it, or is everyone just waiting
> until 64 bit processors become more common?

David Greenman is right.Nevertheless, Peter was planning on
doing the hack, according to his postings to -current.  Please
check the list archives for these things.


> Does FreeBSD use 4M pages exclusively for kernel memory, as in
> Solaris, or is there a more complicated scheme?

FreeBSD starts out using 4K pages for the premapped memory, and
switches over to a 4M page scheme for the initially loaded kernel,
for at least the first 4M.  The PTEs that were for the 4K pages
that are replaced with the 4M mappings are simply lost in the
reload of CR3, and never recovered for the system to use (the
pages containing the PTEs there are "leaked", but it's usually
one page, so 4K is not that bad a leak).

For much of the FreeBSD kernel, 4K pages are used.

I'm pretty sure Solaris also used 4K pages for swappable memory
in the kernel, as well: 4M pages don't make much sense, since
you could, for example, exhaust KVA space with 250 kernel modules
(250 X (1 data + 1 code) * 4M = 2G).



> > If you increase the KVA, then you will decrease the UVA available to
> > user processes.  The total of the two can not exceed 4G.
> 
> In Linux, all of physical memory is mapped into the kernel's virtual
> address space, and hence, until recently Linux was limited to ~3 GB of
> physical memory.  FreeBSD, as I understand, doesn't do that.  So is
> the cause of this limitation that the top half of the kernel has to
> share a virtual address space with user processes?

No.  You need to look at the "copyin" implementation in both OSs to
find the answer.  The way it works is by mapping the address space
of the process in question and the kernel at the same time, and
copying bytes between them.

These are really basic questions about memory layout, which you
should already know the answer to, if you are mucking about in
the KVA size or other parts of the kernel.

I don't know where the Linux limitation comes from; it's really
hard for me to believe "~3G", since it's not an even power of 2,
so I don't really credit this limitation.


> I'll have to read those books one of these days when I have time(6).
> Thanks for the info.

No problem; I think you will have to, if you are planning on
mucking about with more than 4G of physical memory.

-- Terry

To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-stable" in the body of the message




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3CC4C3A0.B1279BFC>