From owner-freebsd-hackers@FreeBSD.ORG Fri Jan 18 16:25:06 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id B081B202; Fri, 18 Jan 2013 16:25:06 +0000 (UTC) (envelope-from alan.l.cox@gmail.com) Received: from mail-la0-f49.google.com (mail-la0-f49.google.com [209.85.215.49]) by mx1.freebsd.org (Postfix) with ESMTP id DBE7D389; Fri, 18 Jan 2013 16:25:05 +0000 (UTC) Received: by mail-la0-f49.google.com with SMTP id fk20so4091650lab.36 for ; Fri, 18 Jan 2013 08:25:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:reply-to:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=bUjt+EpzYBD1d9QBZu1H3cOP8+CaZ0JU8HdUQSJ0bV0=; b=hDhqozb1v8ZrNcN7u2JqfvPLs8MXwgGCUuDQItjAsaxYDRuueG45m2nZVQssf2NPm1 rYZuE9eA0d0qmHPEnbhXmp6j7OI0FNor6pUCFWdxqRbC8t81afyFY7gsAXlEiyDKyEx2 vgKzGktfa+glEFlWEUXQ5CfCPH4QVVGITrGfR+Cazv9vKNS/Bnd8aG1w9zhbKaM89Z7L mow7gbOye1tEpNCjRG9GL2BWwpSj7WJYo5bQg/SbKRumaOvPtp5oAOXhuFf2wY2wMi9u p3ftVh4ZvtuUlkY5pxf/V2YkcyypprsBMabjkJPxQGnbScunRmijmlJxZIU+Cga2XZyE uHMQ== MIME-Version: 1.0 X-Received: by 10.112.14.46 with SMTP id m14mr4013713lbc.98.1358526304638; Fri, 18 Jan 2013 08:25:04 -0800 (PST) Received: by 10.114.21.197 with HTTP; Fri, 18 Jan 2013 08:25:04 -0800 (PST) In-Reply-To: <50F96A67.9080203@freebsd.org> References: <50F96A67.9080203@freebsd.org> Date: Fri, 18 Jan 2013 10:25:04 -0600 Message-ID: Subject: Re: kmem_map auto-sizing and size dependencies From: Alan Cox To: Andre Oppermann Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-hackers , freebsd-current@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: alc@freebsd.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Jan 2013 16:25:06 -0000 I'll follow up with detailed answers to your questions over the weekend. For now, I will, however, point out that you've misinterpreted the tunables. In fact, they say that your kmem map can hold up to 16GB and the current used space is about 58MB. Like other things, the kmem map is auto-sized based on the available physical memory and capped so as not to consume too much of the overall kernel address space. Regards, Alan On Fri, Jan 18, 2013 at 9:29 AM, Andre Oppermann wrote: > The autotuning work is reaching into many places of the kernel and > while trying to tie up all lose ends I've got stuck in the kmem_map > and how it works or what its limitations are. > > During startup the VM is initialized and an initial kernel virtual > memory map is setup in kmem_init() covering the entire KVM address > range. Only the kernel itself is actually allocated within that > map. A bit later on a number of other submaps are allocated (clean_map, > buffer_map, pager_map, exec_map). Also in kmeminit() (in kern_malloc.c, > different from kmem_init) the kmem_map is allocated. > > The (inital?) size of the kmem_map is determined by some voodoo magic, > a sprinkle of nmbclusters * PAGE_SIZE incrementor and lots of tunables. > However it seems to work out to an effective kmem_map_size of about 58MB > on my 16GB AMD64 dev machine: > > vm.kvm_size: 549755809792 > vm.kvm_free: 530233421824 > vm.kmem_size: 16,594,300,928 > vm.kmem_size_min: 0 > vm.kmem_size_max: 329,853,485,875 > vm.kmem_size_scale: 1 > vm.kmem_map_size: 59,518,976 > vm.kmem_map_free: 16,534,777,856 > > The kmem_map serves kernel malloc (via UMA), contigmalloc and everthing > else that uses UMA for memory allocation. > > Mbuf memory too is managed by UMA which obtains the backing kernel memory > from the kmem_map. The limits of the various mbuf memory types have > been considerably raised recently and may make use of 50-75% of all > physically > present memory, or available KVM space, whichever is smaller. > > Now my questions/comments are: > > Does the kmem_map automatically extend itself if more memory is requested? > > Should it be set to a larger initial value based on min(physical,KVM) > space > available? > > The use of nmbclusters for the initial kmem_map size calculation isn't > appropriate anymore due to it being set up later and nmbclusters isn't the > only mbuf relevant mbuf type. We make significant use of page sized mbuf > clusters too. > > The naming and output of the various vm.kmem_* and vm.kvm_* sysctls is > confusing and not easy to reconcile. Either we need some more detailing > more aspects or less. Plus perhaps sysctl subtrees to better describe the > hierarchy of the maps. > > Why are separate kmem submaps being used? Is it to limit memory usage of > certain subsystems? Are those limits actually enforced? > > -- > Andre > ______________________________**_________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/**mailman/listinfo/freebsd-**current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@** > freebsd.org " >