From owner-freebsd-hackers@FreeBSD.ORG Fri Jan 18 16:26:11 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 1076A407; Fri, 18 Jan 2013 16:26:11 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-qc0-f169.google.com (mail-qc0-f169.google.com [209.85.216.169]) by mx1.freebsd.org (Postfix) with ESMTP id 9733D3DA; Fri, 18 Jan 2013 16:26:10 +0000 (UTC) Received: by mail-qc0-f169.google.com with SMTP id t2so2537499qcq.14 for ; Fri, 18 Jan 2013 08:26:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=dyi654dEMV3T/5Id1ivoDxrLLhIsmvWJmuXhkMp/Oe4=; b=ON39WagYTPdzEj0v2rBYkmgZFs14G/RJdBSjXOI5Mln0OTO3b3WyHNMykEj+vSJYXV wrte5gxg0ThQE4nt790GDrF3od8RmqMYuIIMiJbaV4Atfb5bpgeUlAIJjfNTcLaCIYHC K8bvKBiwxkn74rGsbgiFZ4sAYFihwJ9LnhzEfNhrfEZqfUrmmCgbJopmn6wYPKUDqH9F Kv2rb9UD0ITBlPZIqE/Kq9WPhJH9OyYzlagNHCsya9aCf5BbQI0BnDHJr4IgFCw547wm ZsLCKmNapqRrH1IjszPsG2fv3wLmExXpX4Sx4ceAFqFn+ErxGbkVp/6ZqDrwhuxlywAK G0qw== MIME-Version: 1.0 X-Received: by 10.224.17.193 with SMTP id t1mr9796155qaa.89.1358526364372; Fri, 18 Jan 2013 08:26:04 -0800 (PST) Sender: mdf356@gmail.com Received: by 10.229.156.18 with HTTP; Fri, 18 Jan 2013 08:26:04 -0800 (PST) In-Reply-To: <50F96A67.9080203@freebsd.org> References: <50F96A67.9080203@freebsd.org> Date: Fri, 18 Jan 2013 08:26:04 -0800 X-Google-Sender-Auth: cv1G1jW4Z8UYSMJow_1pgYk2gzI Message-ID: Subject: Re: kmem_map auto-sizing and size dependencies From: mdf@FreeBSD.org To: Andre Oppermann Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers , freebsd-current@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Jan 2013 16:26:11 -0000 On Fri, Jan 18, 2013 at 7:29 AM, Andre Oppermann wrote: > The (inital?) size of the kmem_map is determined by some voodoo magic, > a sprinkle of nmbclusters * PAGE_SIZE incrementor and lots of tunables. > However it seems to work out to an effective kmem_map_size of about 58MB > on my 16GB AMD64 dev machine: > > vm.kvm_size: 549755809792 > vm.kvm_free: 530233421824 > vm.kmem_size: 16,594,300,928 > vm.kmem_size_min: 0 > vm.kmem_size_max: 329,853,485,875 > vm.kmem_size_scale: 1 > vm.kmem_map_size: 59,518,976 > vm.kmem_map_free: 16,534,777,856 > > The kmem_map serves kernel malloc (via UMA), contigmalloc and everthing > else that uses UMA for memory allocation. > > Mbuf memory too is managed by UMA which obtains the backing kernel memory > from the kmem_map. The limits of the various mbuf memory types have > been considerably raised recently and may make use of 50-75% of all > physically > present memory, or available KVM space, whichever is smaller. > > Now my questions/comments are: > > Does the kmem_map automatically extend itself if more memory is requested? Not that I recall. > Should it be set to a larger initial value based on min(physical,KVM) space > available? It needs to be smaller than the physical space, because the only limit on the kernel's use of (pinned) memory is the size of the map. So if it is too large there is nothing to stop the kernel from consuming all available memory. The lowmem handler is called when running out of virtual space only (i.e. a failure to allocate a range in the map). > The naming and output of the various vm.kmem_* and vm.kvm_* sysctls is > confusing and not easy to reconcile. Either we need some more detailing > more aspects or less. Plus perhaps sysctl subtrees to better describe the > hierarchy of the maps. > > Why are separate kmem submaps being used? Is it to limit memory usage of > certain subsystems? Are those limits actually enforced? I mostly know about memguard, since I added memguard_fudge(). IIRC some of the submaps are used. The memguard_map specifically is used to know whether an allocation is guarded or not, so at free(9) it can be handled as normal malloc() or as memguard. Cheers, matthew