From owner-freebsd-hackers Sun Jan 26 21:20:40 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3F23837B401 for ; Sun, 26 Jan 2003 21:20:38 -0800 (PST) Received: from ms-smtp-02.southeast.rr.com (ms-smtp-02.southeast.rr.com [24.93.67.83]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2D05C43F13 for ; Sun, 26 Jan 2003 21:20:32 -0800 (PST) (envelope-from bts@fake.com) Received: from mail3.nc.rr.com (fe3 [24.93.67.50]) by ms-smtp-02.southeast.rr.com (8.12.5/8.12.2) with ESMTP id h0R5JMo5022354; Mon, 27 Jan 2003 00:19:25 -0500 (EST) Received: from this.is.fake.com ([24.74.172.220]) by mail3.nc.rr.com with Microsoft SMTPSVC(5.5.1877.757.75); Mon, 27 Jan 2003 00:18:55 -0500 Received: by this.is.fake.com (Postfix, from userid 111) id 9BD38BBC8; Mon, 27 Jan 2003 00:19:48 -0500 (EST) Content-Type: text/plain; charset="iso-8859-1" From: "Brian T. Schellenberger" To: "Sean Hamilton" , Subject: Re: Random disk cache expiry Date: Mon, 27 Jan 2003 00:19:41 -0500 User-Agent: KMail/1.4.2 References: <000501c2c4dd$f43ed450$16e306cf@slugabed.org> <3E34A6BB.2090601@acm.org> <001801c2c5c0$5666de10$16e306cf@slugabed.org> In-Reply-To: <001801c2c5c0$5666de10$16e306cf@slugabed.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Message-Id: <200301270019.44066.bschellenberger@nc.rr.com> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sunday 26 January 2003 11:55 pm, Sean Hamilton wrote: | ----- Original Message ----- | From: "Tim Kientzle" | | | Cycling through large data sets is not really that uncommon. | | I do something like the following pretty regularly: | | find /usr/src -type f | xargs grep function_name | | | | Even scanning through a large dataset once can really hurt | | competing applications on the same machine by flushing | | their data from the cache for no gain. I think this | | is where randomized expiration might really win, by reducing the | | penalty for disk-cache-friendly applications who are competing | | with disk-cache-unfriendly applications. | | | After further thought, I propose something much simpler: when the | kernel is hinted that access will be sequential, it should stop | caching when there is little cache space available, instead of | throwing away old blocks, or be much more hesitant to throw away old | blocks. This to me is imminently sensible. In fact there seem like two rules that have come up in this discussion: 1. For sequential access, you should be very hesitant to throw away *another* processes blocks, at least once you have used more than, say, 25% of the cache or potential cache. 2. For sequential access, you should stop caching before you throw away your own blocks. If it's sequential it is, it seems to me, always a lose to throw away your *own* processes older bllocks on thee same file. These algorithmic changes seem to me to be more likely to be optimal most of the time. A random approach *does* reduce the penalty for worst-case scenarios but at the cost of reducing the benefit of both "normal" and "best-case" scenarios, it seems to me, even more. | Consider that in almost all cases where access is sequential, | as reading continues, the chances of the read being aborted increase: | ie, users downloading files, directory tree traversal, etc. Since the | likelihood of the first byte reading the first byte is very high, and | the next one less high, and the next less yet, etc, it seems to make | sense to tune the caching algorithm to accomodate this. | | While discussing disks, I have a minor complaint: at least on IDE | systems, when doing something like an untar, the entire system is | painfully unresponsive, even though CPU load is low. I presume this | is because when an executable is run, it needs to sit and wait for | the disk. Wouldn't it make sense to give very high disk priority to | executables? Isn't that worth the extra seeks? | | sh | | | To Unsubscribe: send mail to majordomo@FreeBSD.org | with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message