Date: Mon, 20 Aug 2012 09:07:12 -0700 From: =?ISO-8859-1?Q?Gezeala_M=2E_Bacu=F1o_II?= <gezeala@gmail.com> To: Alan Cox <alc@rice.edu> Cc: alc@freebsd.org, freebsd-performance@freebsd.org, Andrey Zonov <andrey@zonov.org>, kib@freebsd.org Subject: Re: vm.kmem_size_max and vm.kmem_size capped at 329853485875 (~307GB) Message-ID: <CAJKO3mXPZVhLo=si%2BEoFPGD5R_m297xedRFY-0N__WOsZBaiCA@mail.gmail.com> In-Reply-To: <50325634.7090904@rice.edu> References: <CAJKO3mU8bfn=jmWNSpvAXOR1AWyAAM0Sio1D1PnOYg8P59V9cg@mail.gmail.com> <CAGH67wS=jue7%2B92jSCyaydOLHC=hPwtndV64FVtC7nhDsPvFng@mail.gmail.com> <CAGH67wTNfW45pgJ_%2BVn_sX%2BP9M5B5wzPT9270dRmWjYF6KerrA@mail.gmail.com> <B74BE4AB-AB67-45BD-BFC3-9AE33A85751C@gmail.com> <502DEAD9.6050304@zonov.org> <CAJKO3mVWOFa9Cby_EWsf_OFHux7YBGSV7aGYSP2YANeJkqZtoQ@mail.gmail.com> <CAJKO3mU1NdkQwNSEDk3wWyLN700=dQ0_jSXt_sx-ABpywNjfsg@mail.gmail.com> <502EB081.3030801@rice.edu> <CAJKO3mWEXUvLtdSvmjgNhhyVqw4j0DuTYm9MqLd9=i9==WLAaA@mail.gmail.com> <502FE98E.40807@rice.edu> <CAJKO3mVUMRfkUpSuk0fDdnEMc3hr087iH5u8b5N60CnPs-gP1g@mail.gmail.com> <50325634.7090904@rice.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, Aug 20, 2012 at 8:22 AM, Alan Cox <alc@rice.edu> wrote:
> On 08/18/2012 19:57, Gezeala M. Bacu=F1o II wrote:
>>
>> On Sat, Aug 18, 2012 at 12:14 PM, Alan Cox<alc@rice.edu> wrote:
>>>
>>> On 08/17/2012 17:08, Gezeala M. Bacu=F1o II wrote:
>>>>
>>>> On Fri, Aug 17, 2012 at 1:58 PM, Alan Cox<alc@rice.edu> wrote:
>>>>>
>>>>> vm.kmem_size controls the maximum size of the kernel's heap, i.e., th=
e
>>>>> region where the kernel's slab and malloc()-like memory allocators
>>>>> obtain
>>>>> their memory. While this heap may occupy the largest portion of the
>>>>> kernel's virtual address space, it cannot occupy the entirety of the
>>>>> address
>>>>> space. There are other things that must be given space within the
>>>>> kernel's
>>>>> address space, for example, the file system buffer map.
>>>>>
>>>>> ZFS does not, however, use the regular file system buffer cache. The
>>>>> ARC
>>>>> takes its place, and the ARC abuses the kernel's heap like nothing
>>>>> else.
>>>>> So, if you are running a machine that only makes trivial use of a
>>>>> non-ZFS
>>>>> file system, like you boot from UFS, but store all of your data in ZF=
S,
>>>>> then
>>>>> you can dramatically reduce the size of the buffer map via boot loade=
r
>>>>> tuneables and proportionately increase vm.kmem_size.
>>>>>
>>>>> Any further increases in the kernel virtual address space size will,
>>>>> however, require code changes. Small changes, but changes nonetheles=
s.
>>>>>
>>>>> Alan
>>>>>
<<snip>>
>>> Your objective should be to reduce the value of "sysctl vfs.maxbufspace=
".
>>> You can do this by setting the loader.conf tuneable "kern.maxbcache" to
>>> the
>>> desired value.
>>>
>>> What does your machine currently report for "sysctl vfs.maxbufspace"?
>>>
>> Here you go:
>> vfs.maxbufspace: 54967025664
>> kern.maxbcache: 0
>
>
> Try setting kern.maxbcache to two billion and adding 50 billion to the
> setting of vm.kmem_size{,_max}.
>
>
Thank you. We'll try this and post back results.
>> Other (probably) relevant values:
>> vfs.hirunningspace: 16777216
>> vfs.lorunningspace: 11206656
>> vfs.bufdefragcnt: 0
>> vfs.buffreekvacnt: 2
>> vfs.bufreusecnt: 320149
>> vfs.hibufspace: 54966370304
>> vfs.lobufspace: 54966304768
>> vfs.maxmallocbufspace: 2748318515
>> vfs.bufmallocspace: 0
>> vfs.bufspace: 10490478592
>> vfs.runningbufspace: 0
>>
>> Let me know if you need other tuneables or sysctl values. Thanks a lot
>> for looking into this.
>>
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJKO3mXPZVhLo=si%2BEoFPGD5R_m297xedRFY-0N__WOsZBaiCA>