Date: Tue, 14 May 1996 21:55:42 -0500 (CDT) From: Joe Greco <jgreco@brasil.moneng.mei.com> To: dyson@freebsd.org Cc: hackers@freebsd.org Subject: Re: A question for the VM gurus..! Message-ID: <199605150255.VAA11471@brasil.moneng.mei.com> In-Reply-To: <199605150029.TAA13410@dyson.iquest.net> from "John S. Dyson" at May 14, 96 07:29:58 pm
next in thread | previous in thread | raw e-mail | index | archive | help
Hi John, > Mincore is supported in current (but only approximately correctly -- it returns > which pages are mapped in a process, not actual residency.) It says in essence, > which pages don't have to be faulted... I have been planning on fixing that. I think I'm going to have to reveal my stupidity here. :-( I do not understand your statement. I would assume that "pages [that] are mapped in a process... [and] ...don't have to be faulted" would necessarily have to be resident. I have (perhaps) a simplistic understanding of how a VM system works. You have a table of pages, which your memory manager arranges into a virtual address space. Accesses to a resident page resolve to a physical memory address thanks to the MMU. Accesses to a nonresident (virtual) page results in a fault, trapped into the OS, the OS fetches the page (from a file, swap, executable, whatever) into a free page, and updates the MMU page table, and you now access it as a resident page. Perhaps a gross oversimplification, I realize, but I am trying to understand your statement, and from my understanding, it is a contradiction :-) If mincore returns "which pages are mapped in a process, not actual residency", I would expect that mincore reports true for every page in the process... (I think this is the statement I am misunderstanding). However, I would expect that "it says in essence which pages don't have to be faulted". Now I'm confused. :-) Maybe it's my SunOS background that is frying my brain. Clarification or a reference to some documentation or a good book on the topic would be appreciated ;-) > Perhaps we should implement a more fully featured mincore -- I did some > mods for Ron Minnich, and since we have a full byte, we would have 8 bits > of info about the pages that we could present :-). I am sure that the > API gods would have something to say about that though. (I never liked > rules, but they are necessary.) > > Madvise is on my list (and probably only a few hours of work to do correctly.) > If you bug me right before the weekend -- I was planning on going to the Dayton > Hamfest, but I don't think I have time now -- I might be able to put something > into -current over the weekend... This is a squeaky wheel situation :-). Understood.. well, let me explain more fully what I am looking at. Usenet news imposes some really interesting challenges on a system engineer. Maintaining a few million articles of varying lengths and having a few hundred clients simultaneously accessing it is a nifty puzzle. Each reader gets a separate process, and you rapidly consume all available resources :-) Now given 200 readers and 2,000,000 articles, the likelihood that any particular reader will access an article that someone else has recently read is fairly low. This data is ripe for being discarded ASAP, and it would be handy to tag these pages as "noncacheable" or "cacheable but VERY discardable". True, currently file accesses are file the open()/close() interface, but it is easy to mmap() the articles instead. Further, I can even mark the pages as MADV_SEQUENTIAL (only useful on large articles I would think), although I don't know how useful this hint would be to the VM system. My other "pet project" requires a functional mincore() - the history file on a large news server may be 150-200MB, and I would like to create a daemon to handle history lookup requests. The file can be mapped and marked with MADV_RANDOM, and when a request comes in requiring a particular bit of data, pages that are found to be !mincore() can be marked with MADV_WILLNEED to ask the VM system to bring them in, while the code goes on to service other requests. I am trying to allow the process to spin through its connection list as rapidly as possible, and with 64MB or 128MB RAM, you can hopefully see how this would be very efficient (from an overall viewpoint). ... Joe ------------------------------------------------------------------------------- Joe Greco - Systems Administrator jgreco@ns.sol.net Solaria Public Access UNIX - Milwaukee, WI 414/546-7968
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199605150255.VAA11471>