Date: Mon, 22 Oct 2012 10:20:50 -0500 From: Alan Cox <alc@rice.edu> To: Marcel Moolenaar <marcel@xcllnt.net> Cc: Poul-Henning Kamp <phk@phk.freebsd.dk>, "freebsd-arch@freebsd.org Arch" <freebsd-arch@freebsd.org>, Tim LaBerge <tlaberge@juniper.net>, Jason Evans <jasone@freebsd.org> Subject: Re: Behavior of madvise(MADV_FREE) Message-ID: <50856452.40902@rice.edu> In-Reply-To: <8D2E1B5A-6DD1-49E3-8F55-B3B816449FFB@xcllnt.net> References: <9FEBC10C-C453-41BE-8829-34E830585E90@xcllnt.net> <4835.1350062021@critter.freebsd.dk> <E6A52D27-0D6A-4175-9ECA-ADE25BFF35C2@xcllnt.net> <F71ACE9D-297E-4565-BB8D-D95D46D90708@freebsd.org> <F67D539D-8BE3-4817-8466-C76DE43AE252@xcllnt.net> <5082F0F3.1070102@rice.edu> <8D2E1B5A-6DD1-49E3-8F55-B3B816449FFB@xcllnt.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On 10/20/2012 21:33, Marcel Moolenaar wrote:
> On Oct 20, 2012, at 11:44 AM, Alan Cox<alc@rice.edu> wrote:
>>> Also, moving the complexity of exactly which hint to give the
>>> kernel under different scenarios isn't that appealing at all.
>>> It just doesn't scale.
>>
>> I think that you're being a bit too pessimistic here. If your use case really corresponds to "this memory is free and will not be reused (or reallocated for a very long time)", then that is qualitatively very different from the way malloc(3) uses MADV_FREE. malloc(3)'s use of MADV_FREE is highly speculative. It doesn't really know what the application is going to do in the future. I don't think that having two distinct hints that distinguish between "speculative" and "non-speculative" uses would be problematic. The distinction is real and also easy to explain. The only danger is that application writers really don't understand their application and use the wrong hint.
> Maybe. I need to think about this. On the surface it's hard to
> belief that any allocator can reliably predict the future, so
> all hints are speculative in that sense. I do buy into the fact
> that malloc(3) has no a priori knowledge of the behavior of an
> application and an application with a special-purpose allocator
> has an allocator with more knowledge of the behavior. I'm just
> not sure this warrants different hints.
>
> I agree that the more complicated the hints the more likely they
> are not being used at all or they are used the wrong way.
>
>>> ... If some VM changes warrant a new hint
>>> to madvise(), you may end up changing multiple daemons. It
>>> seems better to have just 1 hint (i.e. MADV_FREE) and have the
>>> kernel change its behaviour depending on the situation. When
>>> there's plenty of memory, you may even ignore the hint. Under
>>> severe memory pressure you may want to free up the page right
>>> away so that you can give it to some thread that's waiting
>>> for a page.
>>
>> How is this really different from the existing behavior? If a thread is waiting for a page, then the page daemon is running. In particular, it is moving pages from the head of the inactive queue, where they were placed by MADV_FREE, to the cache/free queue and waking up the waiting thread when the aggregate cache/free target is met.
> What we see with FreeBSD 6.1 is that memory remains inactive
> indefinitely. If the behaviour has changed in more recent
> versions, then we'll reap the benefits soon. If not, then we
> (= Juniper) may want to look into this.
You might have a look at the commit message for svn revision 172317. It
describes a problem that existed in FreeBSD from 5.x up until 7.0, when
the physical memory allocator was replaced. Effectively, the page
daemon could wind up spinning its wheels.
However, one thing hasn't changed we only move pages from the inactive
queue to the cache queue as required to meet the aggregate target on
cache and free pages. If the demand for memory is static, we won't move
pages from the inactive queue to the cache queue.
>
>>> At the edge of needing to swap, complex algorithms
>>> may be worthwhile -- or maybe not. I don't know.
>>>
>>> This leads to:
>>> 1. Keep MADV_FREE as it behaves in FreeBSD right now or make
>>> it even more sloppy.
>>
>> I'm not sure that I understand what you mean by "sloppy" here. Can you elaborate?
> It's just a sloppy way of saying that the hint can be ignored
> altogether or that we simply mark the page as clean and not
> do anything else. The point was mostly that the performance
> argument is more important.
>
>
>> 2. Have an idle thread that moves inactive pages to the cache
>>> or free queue if they've been inactive for X minutes, for
>>> some tunable X. Have it back off when the pageout daemon
>>> kicks in.
>>
>> The existing page daemon already wakes up periodically and looks around for something to do. In particular, have a look at vm_pageout_page_stats(). That function tries to do something analogous to what you propose. In part, it tries to prevent munmap(2)ed file-backed pages from getting stuck in the active queue.
> I'll take a look. That's good to know.
>
> \begin{disclaimer}
> Juniper's problem is being stuck with an obsolete version of
> FreeBSD and we're likely to look for solutions to problems that
> don't exist anymore in recent versions. Just bear with us for
> a while :-)
> \end{disclaimer}
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?50856452.40902>