From owner-freebsd-arch Wed May 16 8:18: 3 2001 Delivered-To: freebsd-arch@freebsd.org Received: from bdr-xcon.matchlogic.com (mail.matchlogic.com [205.216.147.127]) by hub.freebsd.org (Postfix) with ESMTP id C5F6F37B422 for ; Wed, 16 May 2001 08:17:58 -0700 (PDT) (envelope-from crandall@matchlogic.com) Received: by mail.matchlogic.com with Internet Mail Service (5.5.2653.19) id ; Wed, 16 May 2001 09:17:46 -0600 Message-ID: <5FE9B713CCCDD311A03400508B8B30130828EDA8@bdr-xcln.corp.matchlogic.com> From: Charles Randall To: 'Matt Dillon' , Roger Larsson Cc: Rik van Riel , arch@FreeBSD.ORG, linux-mm@kvack.org, sfkaplan@cs.amherst.edu Subject: RE: on load control / process swapping Date: Wed, 16 May 2001 09:17:21 -0600 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On a related note, we have a process (currently on Solaris, but = possibly moving to FreeBSD) that reads a 26 GB file just once for a database = load. On Solaris, we use the directio() function call to tell the filesystem to bypass the buffer cache for this file descriptor. From the Solaris directio() man page, DIRECTIO_ON The system behaves as though the application is not going to reuse the file data in the near future. In other words, the file data is not cached in the system's memory pages. We found that without this, Solaris was aggressively trying to cache = the huge input file at the expense of database load performance (but we = knew that we'd never access it again). For some applications this is a huge = win (random I/O on a file much larger than memory seems to be another = case). Would there be an advantage to having a similar feature in FreeBSD (if = not already present)? -Charles -----Original Message----- From: Matt Dillon [mailto:dillon@earth.backplane.com] Sent: Tuesday, May 15, 2001 6:17 PM To: Roger Larsson Cc: Rik van Riel; arch@FreeBSD.ORG; linux-mm@kvack.org; sfkaplan@cs.amherst.edu Subject: Re: on load control / process swapping :Are the heuristics persistent?=20 :Or will the first use after boot use the rough prediction?=20 :For how long time will the heuristic stick? Suppose it is suddenly = used in :a slightly different way. Like two sequential readers instead of = one... : :/RogerL :Roger Larsson :Skellefte=E5 :Sweden It's based on the VM page cache, so its adaptive over time. I = wouldn't call it persistent, it is nothing more then a simple heuristic that 'normally' throws a page away but 'sometimes' caches it. In = otherwords, you lose some performance on the frontend in order to gain some = later on. If you loop through a file enough times, most of the file winds up getting cached. It's still experimental so it is only lightly tied into the system. It seems to work, though, so at some point in the future I'll probably try to put some significant = prediction in. But as I said, it's a very difficult thing to predict. You = can't just put your foot down and say 'I'll cache X amount of file Y'. = That doesn't work at all. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message