Date: Wed, 9 Aug 2006 10:44:31 +1000 From: Q <qdolan@gmail.com> To: Eric Anderson <anderson@centtech.com> Cc: freebsd-fs@freebsd.org Subject: Re: Problem with ufs not releasing vm_pages on busy volume. (soft updates related) Message-ID: <B1C737F3-28C3-48C4-B81B-1ED90E39222A@gmail.com> In-Reply-To: <44D8B6D6.6070209@centtech.com> References: <A05A5F4D-27B0-410B-A32D-0432D22D5A00@gmail.com> <06530464-7491-4BC1-97C7-87285B03F2EB@gmail.com> <44D8B6D6.6070209@centtech.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 09/08/2006, at 2:07 AM, Eric Anderson wrote:
> On 08/08/06 00:14, Q wrote:
>> On 02/08/2006, at 8:10 PM, Q wrote:
>>> I have a problem that seems to be related to something ufs
>>> related not releasing some vm_pages on busy filesystems. I have
>>> two servers running PostgreSQL, one running 6.0-RELEASE, the
>>> other 6.1-RELEASE. Both are under the same (fairly heavy) load,
>>> performing the same operations in bursts every five minutes. The
>>> filesystems in question are 450-500Gig, each server using a
>>> different brand of RAID card, they both have soft-updates enabled.
>>>
>>> The problem is that both servers are seeing an accumulation of
>>> about 100Mb of active pages per day (looking at
>>> vm.stats.vm.v_active_count) that never get released. The only
>>> way to release these pages is to unmount the filesystem and
>>> remount it. Failing to do this results in the server eventually
>>> locking up.
>>>
>>> If someone could provide me with some direction on how to go
>>> about tracking down what might be causing this to happen it
>>> would be much appreciated.
>> I have narrowed the cause of this issue down further to something
>> to do with soft updates. If I turn off soft updates for the
>> filesystem hosting the database the system no longer accumulates
>> active vm_pages constantly. Instead for accumulating 100Mb a day
>> of active vm pages until all memory is consumed, it will hover
>> around 50-60Mb with soft updates disabled.
>> If someone familiar with the softupdates code is willing to help
>> me pinpoint the cause of this problem it would be much appreciated.
>
>
> Is it possible for you to upgrade to the latest 6-STABLE branch,
> just to make sure that the issue hasn't been fixed already?
I did a buildworld on one the machine running 6.0-RELEASE last night.
I just have to schedule some downtime to do the upgrade sometime
today. Having two identical servers has it's advantages.
> Is there any way to reproduce this on another box for testing? (I
> assume not, due to the nature of these things)
I think this bug is very circumstance specific, and my database
design just happens to exercise the bug.
The fact that I have two identical servers experiencing exactly the
same problem does helps narrow the field of possibilities, but the
servers are my no means "expendable".
> Also - I wonder if doing a snapshot on the filesystem would flush
> out the pages - is that something you can try?
Possibly. I will see what I can do.
--
Seeya...Q
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
_____ / Quinton Dolan - qdolan@gmail.com
__ __/ / / __/ / /
/ __ / _/ / / Gold Coast, QLD, Australia
__/ __/ __/ ____/ / - / Ph: +61 419 729 806
_______ /
_\
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?B1C737F3-28C3-48C4-B81B-1ED90E39222A>