From owner-freebsd-net@FreeBSD.ORG Fri Sep 6 23:36:13 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 5AEE7B74; Fri, 6 Sep 2013 23:36:13 +0000 (UTC) (envelope-from hiren.panchasara@gmail.com) Received: from mail-ee0-x234.google.com (mail-ee0-x234.google.com [IPv6:2a00:1450:4013:c00::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C620B26FA; Fri, 6 Sep 2013 23:36:12 +0000 (UTC) Received: by mail-ee0-f52.google.com with SMTP id c41so1894401eek.39 for ; Fri, 06 Sep 2013 16:36:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=YJkdJPw9YCklntanhleYd8ZX+TWmZ/fX0KJtM8YbWNo=; b=DNIEsSESXKfyAHTT5kbxOaDYfjlWI+LsQ5UqKDNeWaiTQMjwwnMQtPgxSQVjRpTqVL e7wMi1+FFV7eLBwnozHCNT445WlZQumyI2LPAREevmUbxkveOglLT7DGuAHL/jezll+L fmFqQ199+bWIM11xP5maUItPhYM0mxWMYOa1BbYs7sNllPQ+/3wSW9GpWsB0Tthd+zPP BOtj1zdixlgcVVOAXsmP1kWeEKLaw/ni/Gr7gY6NEF3M8hcl0rkEzv2txO3UVsecnYbl qV9kTS6yteVfxgm0KTSdQXGDyokCHQZXIFiFheaijeV3njMeVcPsuW4363ISihOQAWAe LLoA== MIME-Version: 1.0 X-Received: by 10.14.126.73 with SMTP id a49mr8091176eei.48.1378510569830; Fri, 06 Sep 2013 16:36:09 -0700 (PDT) Received: by 10.14.105.137 with HTTP; Fri, 6 Sep 2013 16:36:09 -0700 (PDT) Date: Fri, 6 Sep 2013 16:36:09 -0700 Message-ID: Subject: mbuf autotuning effect From: hiren panchasara To: "freebsd-net@freebsd.org" , "freebsd-mips@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Sep 2013 23:36:13 -0000 We are seeing an interesting thing on a mips board with 32MB ram. We run out of mbuf very easily and looking at numbers it seems we are only getting 6mb of maxmbufmem. # sysctl -a | grep hw | grep mem hw.physmem: 33554432 hw.usermem: 21774336 hw.realmem: 33554432 # # sysctl -a | grep maxmbuf kern.ipc.maxmbufmem: 6291456 I believe that number is very low for a board with 32mb of ram. Looking at the code: sys/kern/kern_mbuf.c : tunable_mbinit() 124 realmem = qmin((quad_t)physmem * PAGE_SIZE, vm_kmem_size); 125 maxmbufmem = realmem / 2; 126 TUNABLE_QUAD_FETCH("kern.ipc.maxmbufmem", &maxmbufmem); 127 if (maxmbufmem > realmem / 4 * 3) 128 maxmbufmem = realmem / 4 * 3; So, realmem plays important role in determining maxmbufmem. physmem = 32mb PAGE_SIZE = 4096 vm_kmem_size is calculated inside sys/kern/kern_malloc.c : kmeminit() 705 vm_kmem_size = VM_KMEM_SIZE + nmbclusters * PAGE_SIZE; 706 mem_size = cnt.v_page_count; 707 708 #if defined(VM_KMEM_SIZE_SCALE) 709 vm_kmem_size_scale = VM_KMEM_SIZE_SCALE; 710 #endif 711 TUNABLE_INT_FETCH("vm.kmem_size_scale", &vm_kmem_size_scale); 712 if (vm_kmem_size_scale > 0 && 713 (mem_size / vm_kmem_size_scale) > (vm_kmem_size / PAGE_SIZE)) 714 vm_kmem_size = (mem_size / vm_kmem_size_scale) * PAGE_SIZE; here, VM_KMEM_SIZE = 12*1024*1024 nmbclusters = 0 (initially) PAGE_SIZE = 4096 # sysctl -a | grep v_page_count vm.stats.vm.v_page_count: 7035 and VM_KMEM_SIZE_SCALE = 3 for mips. So, vm_kmem_size = 12mb. Going back to tunable_mbinit(), we get realmem = 12mb. and masmbufmem = 6mb. Wanted to see if I am following the code correctly and how autotuning should work here. cheers, Hiren