From owner-freebsd-hackers Thu Sep 23 9:55:41 1999 Delivered-To: freebsd-hackers@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id BAD6A14E6E for ; Thu, 23 Sep 1999 09:55:39 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id JAA28326; Thu, 23 Sep 1999 09:54:37 -0700 (PDT) (envelope-from dillon) Date: Thu, 23 Sep 1999 09:54:37 -0700 (PDT) From: Matthew Dillon Message-Id: <199909231654.JAA28326@apollo.backplane.com> To: Kevin Day Cc: toasty@dragondata.com (Kevin Day), dcs@newsguy.com (Daniel C. Sobral), hackers@FreeBSD.ORG Subject: Re: Idea: disposable memory References: <199909231433.JAA61714@celery.dragondata.com> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG :I'm now playing with compressed data streams. The decompression is slow, so :I'd like to cache the *decompressed* version of these files. I end up :allocating large amounts of ram in one process to cache the decompressed :data. This is a disavantage over the above scenario, since now the system :swaps out my decompressed data when more ram is needed elsewhere. Swapping :out then swapping back in my decompressed data is about 4x slower than just :re-reading my compressed stream and decompressing it again. : :Why don't I just allocate a predefined amount of memory and use that for a :cache all the time? Most of the time we have about 20MB free on our system. :Sometimes we end up with about 2MB free though, and what's happening now is :that I start paging out data that I could recreate in less time than the :page-in/page-out takes. Hmm. Well, you can check whether the memory has been swapped out with mincore(), and then MADV_FREE it to get rid of it (MADV_FREE'ing something that has been swapped out frees the swap and turns it back into zero-fill). That doesn't get rid of the swapout bandwidth, though. I think, ultimately, you need to manage the memory used for your cache manually. That means using mlock() and munlock() to lock your cache into memory. For example, choose a cache size that you believe the system can support without going bonkers, like 5MB. mmap() 5MB of ram and mlock() it into memory. From that point on until you munlock() it or exit, the memory will not be swapped out. If the purpose of the box is to maintain the flow of video, then the cache is a critical resource and should be treated as such. -Matt Matthew Dillon :Kevin To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message