From owner-freebsd-net@FreeBSD.ORG Sat Sep 7 19:21:50 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 3773FC92; Sat, 7 Sep 2013 19:21:50 +0000 (UTC) (envelope-from hiren.panchasara@gmail.com) Received: from mail-ee0-x22e.google.com (mail-ee0-x22e.google.com [IPv6:2a00:1450:4013:c00::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 68BD42085; Sat, 7 Sep 2013 19:21:49 +0000 (UTC) Received: by mail-ee0-f46.google.com with SMTP id c13so1861778eek.33 for ; Sat, 07 Sep 2013 12:21:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=MXDiKYPL9UTA75YEcSySs+P7pJZDfsKO9VVvYqTraSU=; b=AanHypU5gnUwh6rmIizsShEUbPN9a/h8/DE1eQtud2FcQ8eQwoJe9pg8VH3lIBsEgO OUJGfnXKN9Xq835iJmwSVWmvgPWWK1v5aePaGME4cMsMffkK1xWz7l0lXqHHE5VgqcCc WiQGOaZWgRMbj5PK4Md6/oElE5gl4jbuS6s+8DWrQutCzREc0WEMUvhk27hZoBPAfvwv ovJcfiW64NoqyBiJXwtVbbt8X+0yMzpPz4EURxfZrta28Cgpqtcu/l+uPudPoltZUEMm 5PVHW+CpJ7w34ehClEgiFSWcf4X1kvqXlYMyCNBDobRIqJTEKbKHAeomsTUhTCCyStHF gBDw== MIME-Version: 1.0 X-Received: by 10.14.103.69 with SMTP id e45mr6825732eeg.51.1378581707816; Sat, 07 Sep 2013 12:21:47 -0700 (PDT) Received: by 10.14.105.137 with HTTP; Sat, 7 Sep 2013 12:21:47 -0700 (PDT) Received: by 10.14.105.137 with HTTP; Sat, 7 Sep 2013 12:21:47 -0700 (PDT) In-Reply-To: <9CBFAD35-D651-4E28-BEBB-DC3717F38567@bsdimp.com> References: <9CBFAD35-D651-4E28-BEBB-DC3717F38567@bsdimp.com> Date: Sat, 7 Sep 2013 12:21:47 -0700 Message-ID: Subject: Re: mbuf autotuning effect From: hiren panchasara To: Warner Losh Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-net@freebsd.org, Adrian Chadd , "freebsd-mips@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Sep 2013 19:21:50 -0000 On Sep 6, 2013 8:26 PM, "Warner Losh" wrote: > > > On Sep 6, 2013, at 7:11 PM, Adrian Chadd wrote: > > > Yeah, why is VM_KMEM_SIZE only 12mbyte for MIPS? That's a little low for a > > platform that has a direct map that's slightly larger than 12mb :) > > > > Warner? Juli? > > All architectures have it at 12MB, except sparc64 where it is 16MB. This can be changed with the options VM_KMEM_SIZE=xxxxx in the config file. Right. Does that mean for any platform, if we do not have nmbclusters pre-set in kmeminit() than we will always have pretty low value of vm_kmem_size. And because of that, if maxmbufmem is not pre-set (via loader.conf) inside tunable_mbinit() , we will have very low value for maxmbufmem too. I hope (partially believe) that my understanding is not entirely correct. Because if its correct, we arw depending on loader.conf instead of actually auto tuning. Thanks, Hiren > > So my guess as to why this is the case: cut and paste worked, so nobody changed it after that. > > # Still need to reads hiren's email to comprehend it... > > Warner > > > > > > > > -adrian > > > > > > > > On 6 September 2013 16:36, hiren panchasara wrote: > > > >> We are seeing an interesting thing on a mips board with 32MB ram. > >> > >> We run out of mbuf very easily and looking at numbers it seems we are only > >> getting 6mb of maxmbufmem. > >> > >> # sysctl -a | grep hw | grep mem > >> hw.physmem: 33554432 > >> hw.usermem: 21774336 > >> hw.realmem: 33554432 > >> # > >> # sysctl -a | grep maxmbuf > >> kern.ipc.maxmbufmem: 6291456 > >> > >> I believe that number is very low for a board with 32mb of ram. > >> > >> Looking at the code: > >> > >> sys/kern/kern_mbuf.c : tunable_mbinit() > >> > >> 124 realmem = qmin((quad_t)physmem * PAGE_SIZE, vm_kmem_size); > >> 125 maxmbufmem = realmem / 2; > >> 126 TUNABLE_QUAD_FETCH("kern.ipc.maxmbufmem", &maxmbufmem); > >> 127 if (maxmbufmem > realmem / 4 * 3) > >> 128 maxmbufmem = realmem / 4 * 3; > >> > >> So, realmem plays important role in determining maxmbufmem. > >> > >> physmem = 32mb > >> PAGE_SIZE = 4096 > >> > >> vm_kmem_size is calculated inside sys/kern/kern_malloc.c : kmeminit() > >> > >> 705 vm_kmem_size = VM_KMEM_SIZE + nmbclusters * PAGE_SIZE; > >> 706 mem_size = cnt.v_page_count; > >> 707 > >> 708 #if defined(VM_KMEM_SIZE_SCALE) > >> 709 vm_kmem_size_scale = VM_KMEM_SIZE_SCALE; > >> 710 #endif > >> 711 TUNABLE_INT_FETCH("vm.kmem_size_scale", &vm_kmem_size_scale); > >> 712 if (vm_kmem_size_scale > 0 && > >> 713 (mem_size / vm_kmem_size_scale) > (vm_kmem_size / > >> PAGE_SIZE)) > >> 714 vm_kmem_size = (mem_size / vm_kmem_size_scale) * > >> PAGE_SIZE; > >> > >> here, > >> VM_KMEM_SIZE = 12*1024*1024 > >> nmbclusters = 0 (initially) > >> PAGE_SIZE = 4096 > >> # sysctl -a | grep v_page_count > >> vm.stats.vm.v_page_count: 7035 > >> > >> and VM_KMEM_SIZE_SCALE = 3 for mips. > >> > >> So, vm_kmem_size = 12mb. > >> > >> Going back to tunable_mbinit(), > >> we get realmem = 12mb. > >> and masmbufmem = 6mb. > >> > >> > >> Wanted to see if I am following the code correctly and how autotuning > >> should work here. > >> > >> cheers, > >> Hiren > >> _______________________________________________ > >> freebsd-net@freebsd.org mailing list > >> http://lists.freebsd.org/mailman/listinfo/freebsd-net > >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > >> > > _______________________________________________ > > freebsd-mips@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-mips > > To unsubscribe, send any mail to "freebsd-mips-unsubscribe@freebsd.org" >